Bug 1509962: Vendor redo==2.0.2; r=mtabara

Differential Revision: https://phabricator.services.mozilla.com/D12973

--HG--
extra : moz-landing-system : lando

third_party/python/redo/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 1.0
 Name: redo
-Version: 1.6
+Version: 2.0.2
 Summary: Utilities to retry Python callables.
 Home-page: https://github.com/bhearsum/redo
 Author: Ben Hearsum

third_party/python/redo/README

@@ -1,4 +0,0 @@
-Redo - Utilities to retry Python callables
-******************************************
-
-Redo provides various means to add seamless retriability to any Python callable. Redo includes a plain function (redo.retry), a decorator (redo.retriable), and a context manager (redo.retrying) to enable you to integrate it in the best possible way for your project. As a bonus, a standalone interface is also included ("retry"). For details and sample invocations have a look at the docstrings in redo/__init__.py.

third_party/python/redo/README.md

@@ -0,0 +1,147 @@
+
+# Redo - Utilities to retry Python callables
+******************************************
+
+## Introduction
+
+Redo provides various means to add seamless ability to retry to any Python callable. Redo includes a plain function `(redo.retry)`, a decorator `(redo.retriable)`, and a context manager `(redo.retrying)` to enable you to integrate it in the best possible way for your project. As a bonus, a standalone interface is also included `("retry")`.
+
+## Installation
+
+For installing with pip, run following commands
+> pip install redo
+
+## How To Use
+Below is the list of functions available
+* retrier
+* retry
+* retriable
+* retrying (contextmanager)
+
+### retrier(attempts=5, sleeptime=10, max_sleeptime=300, sleepscale=1.5, jitter=1)
+A generator function that sleeps between retries, handles exponential back off and jitter. The action you are retrying is meant to run after retrier yields. At each iteration, we sleep for `sleeptime + random.randint(-jitter, jitter)`. Afterwards sleeptime is multiplied by sleepscale for the next iteration.
+
+**Arguments Detail:**    
+
+1. **attempts (int):** maximum number of times to try; defaults to 5
+2. **sleeptime (float):** how many seconds to sleep between tries; defaults to 60s (one minute)
+3. **max_sleeptime (float):** the longest we'll sleep, in seconds; defaults to 300s (five minutes)
+4. **sleepscale (float):** how much to multiply the sleep time by each iteration; defaults to 1.5
+5. **jitter (int):** random jitter to introduce to sleep time each iteration. the amount is chosen at random between `[-jitter, +jitter]` defaults to 1
+
+**Output:**
+None, a maximum of `attempts` number of times
+
+**Example:**
+
+    >>> n = 0
+    >>> for _ in retrier(sleeptime=0, jitter=0):
+    ...     if n == 3:
+    ...         # We did the thing!
+    ...         break
+    ...     n += 1
+    >>> n
+    3
+    >>> n = 0
+    >>> for _ in retrier(sleeptime=0, jitter=0):
+    ...     if n == 6:
+    ...         # We did the thing!
+    ...         break
+    ...     n += 1
+    ... else:
+    ...     print("max tries hit")
+    max tries hit
+
+### retry(action, attempts=5, sleeptime=60, max_sleeptime=5 * 60, sleepscale=1.5, jitter=1, retry_exceptions=(Exception,), cleanup=None, args=(), kwargs={})  
+Calls an action function until it succeeds, or we give up.
+
+**Arguments Detail:**  
+
+1. **action (callable):** the function to retry
+2. **attempts (int):** maximum number of times to try; defaults to 5
+3. **sleeptime (float):** how many seconds to sleep between tries; defaults to 60s (one minute)
+4. **max_sleeptime (float):** the longest we'll sleep, in seconds; defaults to 300s (five minutes)
+5. **sleepscale (float):** how much to multiply the sleep time by each iteration; defaults to 1.5
+6. **jitter (int):** random jitter to introduce to sleep time each iteration. The amount is chosen at random between `[-jitter, +jitter]` defaults to 1
+7. **retry_exceptions (tuple):** tuple of exceptions to be caught. If other exceptions are raised by `action()`, then these are immediately re-raised to the caller.
+8. **cleanup (callable):** optional; called if one of `retry_exceptions` is caught. No arguments are passed to the cleanup function; if your cleanup requires arguments, consider using `functools.partial` or a `lambda` function.
+9. **args (tuple):** positional arguments to call `action` with
+10. **kwargs (dict):** keyword arguments to call `action` with
+
+**Output:**
+ Whatever action`(*args, **kwargs)` returns
+ 
+ **Output:**
+        Whatever action(*args, **kwargs) raises. `retry_exceptions` are caught
+        up until the last attempt, in which case they are re-raised.
+
+**Example:**
+
+    >>> count = 0
+    >>> def foo():
+    ...     global count
+    ...     count += 1
+    ...     print(count)
+    ...     if count < 3:
+    ...         raise ValueError("count is too small!")
+    ...     return "success!"
+    >>> retry(foo, sleeptime=0, jitter=0)
+    1
+    2
+    3
+    'success!'
+
+### retriable(*retry_args, **retry_kwargs)
+A decorator factory for `retry()`. Wrap your function in `@retriable(...)` to give it retry powers!
+
+**Arguments Detail:**  
+        Same as for `retry`, with the exception of `action`, `args`, and `kwargs`,
+        which are left to the normal function definition.
+
+**Output:**
+A function decorator
+
+**Example:**
+
+    >>> count = 0
+    >>> @retriable(sleeptime=0, jitter=0)
+    ... def foo():
+    ...     global count
+    ...     count += 1
+    ...     print(count)
+    ...     if count < 3:
+    ...         raise ValueError("count too small")
+    ...     return "success!"
+    >>> foo()
+    1
+    2
+    3
+    'success!'
+
+### retrying(func, *retry_args, **retry_kwargs)
+A context manager for wrapping functions with retry functionality.
+
+**Arguments Detail:**   
+
+1. **func (callable):** the function to wrap
+other arguments as per `retry`
+
+**Output:**
+A context manager that returns `retriable(func)` on `__enter__`
+
+**Example:**
+
+    >>> count = 0
+    >>> def foo():
+    ...     global count
+    ...     count += 1
+    ...     print(count)
+    ...     if count < 3:
+    ...         raise ValueError("count too small")
+    ...     return "success!"
+    >>> with retrying(foo, sleeptime=0, jitter=0) as f:
+    ...     f()
+    1
+    2
+    3
+    'success!'

third_party/python/redo/redo.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 1.0
 Name: redo
-Version: 1.6
+Version: 2.0.2
 Summary: Utilities to retry Python callables.
 Home-page: https://github.com/bhearsum/redo
 Author: Ben Hearsum

third_party/python/redo/redo.egg-info/SOURCES.txt

@@ -1,3 +1,4 @@
+README.md
 setup.cfg
 setup.py
 redo/__init__.py
@@ -6,5 +7,4 @@ redo.egg-info/PKG-INFO
 redo.egg-info/SOURCES.txt
 redo.egg-info/dependency_links.txt
 redo.egg-info/entry_points.txt
-redo.egg-info/pbr.json
 redo.egg-info/top_level.txt

third_party/python/redo/redo/__init__.py

@@ -18,18 +18,18 @@ def retrier(attempts=5, sleeptime=10, max_sleeptime=300, sleepscale=1.5, jitter=
     backoff and jitter. The action you are retrying is meant to run after
     retrier yields.
 
-    At each iteration, we sleep for sleeptime + random.randint(-jitter, jitter).
+    At each iteration, we sleep for sleeptime + random.uniform(-jitter, jitter).
     Afterwards sleeptime is multiplied by sleepscale for the next iteration.
 
     Args:
         attempts (int): maximum number of times to try; defaults to 5
         sleeptime (float): how many seconds to sleep between tries; defaults to
-                           60s (one minute)
+                           10 seconds
         max_sleeptime (float): the longest we'll sleep, in seconds; defaults to
                                300s (five minutes)
         sleepscale (float): how much to multiply the sleep time by each
                             iteration; defaults to 1.5
-        jitter (int): random jitter to introduce to sleep time each iteration.
+        jitter (float): random jitter to introduce to sleep time each iteration.
                       the amount is chosen at random between [-jitter, +jitter]
                       defaults to 1
 
@@ -68,9 +68,9 @@ def retrier(attempts=5, sleeptime=10, max_sleeptime=300, sleepscale=1.5, jitter=
         yield sleeptime_real
 
         if jitter:
-            sleeptime_real = sleeptime + random.randint(-jitter, jitter)
+            sleeptime_real = sleeptime + random.uniform(-jitter, jitter)
             # our jitter should scale along with the sleeptime
-            jitter = int(jitter * sleepscale)
+            jitter = jitter * sleepscale
         else:
             sleeptime_real = sleeptime
 
@@ -87,7 +87,7 @@ def retrier(attempts=5, sleeptime=10, max_sleeptime=300, sleepscale=1.5, jitter=
 
 def retry(action, attempts=5, sleeptime=60, max_sleeptime=5 * 60,
           sleepscale=1.5, jitter=1, retry_exceptions=(Exception,),
-          cleanup=None, args=(), kwargs={}):
+          cleanup=None, args=(), kwargs={}, log_args=True):
     """
     Calls an action function until it succeeds, or we give up.
 
@@ -100,7 +100,7 @@ def retry(action, attempts=5, sleeptime=60, max_sleeptime=5 * 60,
                                300s (five minutes)
         sleepscale (float): how much to multiply the sleep time by each
                             iteration; defaults to 1.5
-        jitter (int): random jitter to introduce to sleep time each iteration.
+        jitter (float): random jitter to introduce to sleep time each iteration.
                       the amount is chosen at random between [-jitter, +jitter]
                       defaults to 1
         retry_exceptions (tuple): tuple of exceptions to be caught. If other
@@ -113,6 +113,8 @@ def retry(action, attempts=5, sleeptime=60, max_sleeptime=5 * 60,
                             function.
         args (tuple): positional arguments to call `action` with
         kwargs (dict): keyword arguments to call `action` with
+        log_args (bool): whether or not to include args and kwargs in log
+                         messages. Defaults to True.
 
     Returns:
         Whatever action(*args, **kwargs) returns
@@ -140,17 +142,17 @@ def retry(action, attempts=5, sleeptime=60, max_sleeptime=5 * 60,
     assert not cleanup or callable(cleanup)
 
     action_name = getattr(action, '__name__', action)
-    if args or kwargs:
-        log_attempt_format = ("retry: calling %s with args: %s,"
-                              " kwargs: %s, attempt #%%d"
-                              % (action_name, args, kwargs))
+    if log_args and (args or kwargs):
+        log_attempt_args = ("retry: calling %s with args: %s,"
+                            " kwargs: %s, attempt #%d",
+                            action_name, args, kwargs)
     else:
-        log_attempt_format = ("retry: calling %s, attempt #%%d"
-                              % action_name)
+        log_attempt_args = ("retry: calling %s, attempt #%d",
+                            action_name)
 
     if max_sleeptime < sleeptime:
-        log.debug("max_sleeptime %d less than sleeptime %d" % (
-            max_sleeptime, sleeptime))
+        log.debug("max_sleeptime %d less than sleeptime %d",
+                  max_sleeptime, sleeptime)
 
     n = 1
     for _ in retrier(attempts=attempts, sleeptime=sleeptime,
@@ -158,14 +160,15 @@ def retry(action, attempts=5, sleeptime=60, max_sleeptime=5 * 60,
                      jitter=jitter):
         try:
             logfn = log.info if n != 1 else log.debug
-            logfn(log_attempt_format, n)
+            log_attempt_args += (n, )
+            logfn(*log_attempt_args)
             return action(*args, **kwargs)
         except retry_exceptions:
             log.debug("retry: Caught exception: ", exc_info=True)
             if cleanup:
                 cleanup()
             if n == attempts:
-                log.info("retry: Giving up on %s" % action_name)
+                log.info("retry: Giving up on %s", action_name)
                 raise
             continue
         finally:

third_party/python/redo/redo/cmd.py

@@ -12,8 +12,8 @@ from redo import retrying
 log = logging.getLogger(__name__)
 
 
-def main():
-    from argparse import ArgumentParser
+def main(argv):
+    from argparse import ArgumentParser, REMAINDER
 
     parser = ArgumentParser()
     parser.add_argument(
@@ -26,9 +26,9 @@ def main():
         "-m", "--max-sleeptime", type=int, default=5*60,
         help="Maximum length of time to sleep between attempts (limits backoff length).")
     parser.add_argument("-v", "--verbose", action="store_true", default=False)
-    parser.add_argument("cmd", nargs="+", help="Command to run. Eg: wget http://blah")
+    parser.add_argument("cmd", nargs=REMAINDER, help="Command to run. Eg: wget http://blah")
 
-    args = parser.parse_args()
+    args = parser.parse_args(argv[1:])
 
     if args.verbose:
         logging.basicConfig(level=logging.INFO)
@@ -49,5 +49,6 @@ def main():
         rc = getattr(e, "returncode", -2)
         sys.exit(rc)
 
+
 if __name__ == "__main__":
-    main()
+    main(sys.argv)

third_party/python/redo/setup.cfg

@@ -4,5 +4,4 @@ universal = 1
 [egg_info]
 tag_build = 
 tag_date = 0
-tag_svn_revision = 0
 

third_party/python/redo/setup.py

@@ -6,7 +6,7 @@ except ImportError:
 
 setup(
     name="redo",
-    version="1.6",
+    version="2.0.2",
     description="Utilities to retry Python callables.",
     author="Ben Hearsum",
     author_email="ben@hearsum.ca",

third_party/python/requirements.in

@@ -9,6 +9,7 @@ pipenv==2018.5.18
 psutil==5.4.3
 pytest==3.6.2
 python-hglib==2.4
+redo==2.0.2
 requests==2.9.1
 six==1.10.0
 taskcluster==4.0.1

third_party/python/requirements.txt

@@ -75,6 +75,9 @@ pytest==3.6.2 \
     --hash=sha256:90898786b3d0b880b47645bae7b51aa9bbf1e9d1e4510c2cfd15dd65c70ea0cd
 python-hglib==2.4 \
     --hash=sha256:693d6ed92a6566e78802c7a03c256cda33d08c63ad3f00fcfa11379b184b9462
+redo==2.0.2 \
+    --hash=sha256:29159a717454e1f276c7c509b81357e167a0b9218c68adf8ca8b0499363877ad \
+    --hash=sha256:703603d61b4ae7fa14a9dce3db22d8789284e99be997f558137612e847ead3cb
 requests==2.9.1 \
     --hash=sha256:113fbba5531a9e34945b7d36b33a084e8ba5d0664b703c81a7c572d91919a5b8 \
     --hash=sha256:c577815dd00f1394203fc44eb979724b098f88264a9ef898ee45b8e5e9cf587f