xotl.tools.future.itertools
- Functions creating iterators for efficient looping¶
Several util functions for iterators.
Changed in version 1.8.4: Renamed to xotl.tools.future.iterator
. The
xotl.tools.iterators
is now a deprecated alias.
-
xotl.tools.future.itertools.
dict_update_new
(target, source, fail=False)[source]¶ Update values in
source
that are new (not present) intarget
.If
fail
is True and a value is already set, an error is raised.
-
xotl.tools.future.itertools.
first_non_null
(iterable, default=None)[source]¶ Returns the first value from iterable which is non-null.
This is roughly the same as:
next((x for x in iter(iterable) if x), default)
New in version 1.4.0.
-
xotl.tools.future.itertools.
slides
(iterable: Iterable[T], width: int = 2, fill: X = None) → Iterator[Tuple[Union[T, X, None], ...]][source]¶ Creates a sliding window of a given
width
over an iterable:>>> list(slides(range(1, 11))) [(1, 2), (3, 4), (5, 6), (7, 8), (9, 10)]
If the iterator does not yield a width-aligned number of items, the last slice returned is filled with
fill
(by default None) unlessfill
isNO_FILL
:>>> list(slides(range(1, 11), width=3)) [(1, 2, 3), (4, 5, 6), (7, 8, 9), (10, None, None)] >>> list(slides(range(1, 11), width=3, fill=NO_FILL)) [(1, 2, 3), (4, 5, 6), (7, 8, 9), (10,)]
Changed in version 1.4.0: If the
fill
argument is a collection is cycled over to get the filling, just like infirst_n()
.Changed in version 1.4.2: The
fill
argument now defaults to None, instead of Unset.Changed in version 2.2.2: The
fill
argument can now take the contantNO_FILL
-
xotl.tools.future.itertools.
continuously_slides
(iterable: Iterable[T], width: int = 2, fill: X = None) → Iterator[Tuple[Union[T, X, None], ...]][source]¶ Similar to
slides()
but moves one item at the time (i.e continuously).fill
is only used to fill the fist chunk if theiterable
has less items than thewidth
of the window.Example (generate a texts tri-grams):
>>> slider = continuously_slides(str('maupassant'), 3) >>> list(str('').join(chunk) for chunk in slider) ['mau', 'aup', 'upa', 'pas', 'ass', 'ssa', 'san', 'ant']
-
xotl.tools.future.itertools.
ungroup
(iterator: Iterable[Tuple[X, Iterable[T]]]) → Iterator[T][source]¶ Reverses the operation of
itertools.groupby()
(or similar).The
iterator
should produce pairs of(_, xs)
; wherexs
is another iterator (or iterable).It’s guaranteed that the
iterator
will be consumed at the boundaries of each pair, i.e. before taking another pair(_, ys)
fromiterator
the firstxs
will be fully yielded.Demonstration:
>>> def groups(): ... def chunk(s): ... for x in range(s, s+3): ... print('Yielding x:', x) ... yield x ... ... for g in range(2): ... print('Yielding group', g) ... yield g, chunk(g)
>>> list(ungroup(groups())) Yielding group 0 Yielding x: 0 Yielding x: 1 Yielding x: 2 Yielding group 1 Yielding x: 1 Yielding x: 2 Yielding x: 3 [0, 1, 2, 1, 2, 3]
This is not the same as:
>>> import itertools >>> xs = itertools.chain(*(xs for _, xs in groups())) Yielding group 0 Yielding group 1
Notice that the iterator was fully consumed just to create the arguments to
chain()
.New in version 1.7.3.
-
xotl.tools.future.itertools.
merge
(*iterables, key=None)[source]¶ Merge the iterables in order.
Return an iterator that yields all items from
iterables
following the order given bykey
. Ifkey
is not given we compare the items.If the
iterables
yield their items in increasing order (w.r.tkey
), the result is also ordered (like a merge sort).merge()
returns the empty iterator.New in version 1.8.4.
Changed in version 2.1.0: Based on
heapq.merge()
. In Python 3.5+, this is just an alias of it.Deprecated since version 2.1.0: Use
heapq.merge()
directly. This function will be removed when we support for Python 3.4.
-
xotl.tools.future.itertools.
delete_duplicates
(seq[, key=lambda x: x])[source]¶ Remove all duplicate elements from
seq
.Two items
x
andy
are considered equal (duplicates) ifkey(x) == key(y)
. By defaultkey
is the identity function.Works with any sequence that supports
len()
,__getitem__()
, andaddition
.Note
seq.__getitem__
should work properly with slices.The return type will be the same as that of the original sequence.
New in version 1.5.5.
Changed in version 1.7.4: Added the
key
argument. Clarified the documentation:seq
should also implement the__add__
method and that its__getitem__
method should deal with slices.
-
xotl.tools.future.itertools.
iter_delete_duplicates
(iter[, key=lambda x: x])[source]¶ Yields non-repeating (and consecutive) items from
iter
.key
has the same meaning as indelete_duplicates()
.Examples:
>>> list(iter_delete_duplicates('AAAaBBBA')) ['A', 'a', 'B', 'A']
>>> list(iter_delete_duplicates('AAAaBBBA', key=lambda x: x.lower())) ['A', 'B', 'A']
New in version 1.7.4.
-
xotl.tools.future.itertools.
iter_without_duplicates
(iter[, key=lambda x: x])[source]¶ Yields non-repeating items from
iter
.key
has the same meaning as indelete_duplicates()
.The difference between this function and
iter_delete_duplicates()
is that we ensure the same item (as perkey
) is produced only once; whileiter_delete_duplicates()
only remove consecutive repeating items.Example:
>>> list(iter_without_duplicates('AAAaBBBA', key=lambda x: x.lower())) ['A', 'B']
-
xotl.tools.future.itertools.
flatten
(sequence, is_scalar=xotl.tools.types.is_scalar, depth=None)[source]¶ Flatten-out a sequence.
It takes care of everything deemed a collection (i.e, not a scalar according to the callable passed in
is_scalar
argument; ifNone
, iterables -but strings- will be considered as scalars.For example:
>>> range_ = lambda *a: list(range(*a)) >>> tuple(flatten((1, range_(2, 5), range(5, 10)))) (1, 2, 3, 4, 5, 6, 7, 8, 9)
If
depth
is None the collection is flattened recursively until the “bottom” is reached. Ifdepth
is an integer then the collection is flattened up to that level.depth=0
means not to flatten. Nested iterators are not “exploded” if under the stateddepth
:# In the following doctest we use ``...range(...X)`` because the # string repr of range differs in Py2 and Py3k. >>> tuple(flatten((range_(2), range(2, 4)), depth=0)) # doctest: +ELLIPSIS # noqa ([0, 1], ...range(2, 4)) >>> tuple(flatten((range(2), range_(2, 4)), depth=0)) # doctest: +ELLIPSIS # noqa (...range(...2), [2, 3])
Note
Compatibility issue
In Python 2
bytes
is the standard string but in Python 3 is a binary buffer, soflatten([b'abc', [1, 2, 3]])
will deliver different results.
-
xotl.tools.iterators.
zip
([iter1[, iter2[, ...]]])¶ Return a zip-like object whose
next()
method returns a tuple where the i-th element comes from the i-th iterable argument. Thenext()
method continues until the shortest iterable in the argument sequence is exhausted and then it raises StopIteration.Deprecated since version 2.1.0: Use the builtin
zip()
. This function will be removed in xotl.tools 3.
-
xotl.tools.iterators.
map
(func, *iterables)¶ Make an iterator that computes the function using arguments from each of the iterables. It stops when the shortest iterable is exhausted instead of filling in None for shorter iterables.
Deprecated since version 2.1.0: Use the builtin
map()
. This function will be removed in xotl.tools 3.
-
xotl.tools.iterators.
zip_longest
(*iterables, fillvalue=None)¶ Make an iterator that aggregates elements from each of the iterables. If the iterables are of uneven length, missing values are filled-in with fillvalue. Iteration continues until the longest iterable is exhausted.
If one of the iterables is potentially infinite, then the
zip_longest()
function should be wrapped with something that limits the number of calls (for exampleislice()
ortakewhile()
). If not specified,fillvalue
defaults to None.This function is actually an alias to
itertools.izip_longest()
in Python 2.7, and an alias toitertools.zip_longest()
in Python 3.3.