xotl.tools.future.collections - High-performance container datatypes

This module extends the standard library’s collections. You may use it as a drop-in replacement in many cases.

Avoid importing * from this module since this is different in Python 2.7 and Python 3.3. Notably importing abc is not available in Python 2.7.

We have backported several Python 3.3 features but not all.

class xotl.tools.future.collections.defaultdict[source]

A hack for collections.defaultdict that passes the key and a copy of self as a plain dict (to avoid infinity recursion) to the callable.

Examples:

>>> from xotl.tools.future.collections import defaultdict
>>> d = defaultdict(lambda key, d: 'a')
>>> d['abc']
'a'

Since the second parameter is actually a dict-copy, you may (naively) do the following:

>>> d = defaultdict(lambda k, d: d[k])
>>> d['abc']
Traceback (most recent call last):
    ...
KeyError: 'abc'

You may use this class as a drop-in replacement for collections.defaultdict:

>>> d = defaultdict(lambda: 1)
>>> d['abc']
1
class xotl.tools.future.collections.opendict[source]

A dictionary implementation that mirrors its keys as attributes.

For example:

>>> d = opendict(es='spanish')
>>> d.es
'spanish'

>>> d['es'] = 'espanol'
>>> d.es
'espanol'

Setting attributes not already included does not makes them keys:

>>> d.en = 'English'
>>> set(d)
{'es'}
classmethod from_enum(enumclass)[source]

Creates an opendict from an enumeration class.

If enumclass lacks the __members__ dictionary, take the __dict__ of the class disregarding the keys that cannot be public identifiers. If enumclass has the __members__ attribute this is the same as opendict(enumclass.__members__).

Example:

>>> from xotl.tools.future.collections import opendict
>>> @opendict.from_enum
>>> class Foo:
...    x = 1
...    _y = 2

>>> type(Foo) is opendict
True

>>> dict(Foo)
{'x': 1}
class xotl.tools.future.collections.codedict[source]

A dictionary implementation that evaluate keys as Python expressions.

This is also a open dict (see OpenDictMixin for more info).

Example:

>>> cd = codedict(x=1, y=2, z=3.0)
>>> '{_[x + y]} is 3 --  {_[x + z]} is 4.0'.format(_=cd)
'3 is 3 --  4.0 is 4.0'

It supports the right shift (>>) operator as a format operand (using _ as the special name for the code dict):

>>> cd >> '{_[x + y]} is 3 --  {_[x + z]} is 4.0 -- {x} is 1'
'3 is 3 --  4.0 is 4.0 -- 1 is 1'

It also implements the left shift (<<) operator:

>>> '{_[x + y]} is 3 --  {_[x + z]} is 4.0 -- {x} is 1' << cd
'3 is 3 --  4.0 is 4.0 -- 1 is 1'

New in version 1.8.3.

class xotl.tools.future.collections.Counter(**kwds)[source]

Dict subclass for counting hashable items. Sometimes called a bag or multiset. Elements are stored as dictionary keys and their counts are stored as dictionary values.

>>> c = Counter('abcdeabcdabcaba')  # count elements from a string
>>> c.most_common(3)                # three most common elements
[('a', 5), ('b', 4), ('c', 3)]
>>> sorted(c)                       # list all unique elements
['a', 'b', 'c', 'd', 'e']
>>> ''.join(sorted(c.elements()))   # list elements with repetitions
'aaaaabbbbcccdde'
>>> sum(c.values())                 # total of all counts
15
>>> c['a']                          # count of letter 'a'
5
>>> for elem in 'shazam':           # update counts from an iterable
...     c[elem] += 1                # by adding 1 to each element's count
>>> c['a']                          # now there are seven 'a'
7
>>> del c['b']                      # remove all 'b'
>>> c['b']                          # now there are zero 'b'
0
>>> d = Counter('simsalabim')       # make another counter
>>> c.update(d)                     # add in the second counter
>>> c['a']                          # now there are nine 'a'
9
>>> c.clear()                       # empty the counter
>>> c
Counter()

Note: If a count is set to zero or reduced to zero, it will remain in the counter until the entry is deleted or the counter is cleared:

>>> c = Counter('aaabbc')
>>> c['b'] -= 2                     # reduce the count of 'b' by two
>>> c.most_common()                 # 'b' is still in, but its count is zero
[('a', 3), ('c', 1), ('b', 0)]

Note

Backported from Python 3.3. In Python 3.3 this is an alias.

class xotl.tools.future.collections.OrderedDict[source]

Dictionary that remembers insertion order

Note

Backported from Python 3.3. In Python 3.3 this is an alias.

class xotl.tools.future.collections.OpenDictMixin[source]

A mixin for mappings implementation that expose keys as attributes.

For example:

>>> from xotl.tools.objects import SafeDataItem as safe
>>> class MyOpenDict(OpenDictMixin, dict):
...     __slots__ = safe.slot(OpenDictMixin.__cache_name__, dict)
>>> d = MyOpenDict({'es': 'spanish'})
>>> d.es
'spanish'
>>> d['es'] = 'espanol'
>>> d.es
'espanol'

When setting or deleting an attribute, the attribute name is regarded as key in the mapping if neither of the following condition holds:

  • The name is a slot.
  • The object has a __dict__ attribute and the name is key there.

This mixin defines the following features that can be redefined:

_key2identifier

Protected method, receives a key as argument and must return a valid identifier that is used instead the key as an extended attribute.

__cache_name__

Inner field to store a cached mapping between actual keys and calculated attribute names. The field must be always implemented as a SafeDataItem descriptor and must be of type dict. There are two ways of implementing this:

  • As a slot. The first time of this implementation is an example. Don’t forget to pass the second parameter with the constructor dict.

  • As a normal descriptor:

    >>> from xotl.tools.objects import SafeDataItem as safe
    >>> class MyOpenDict(OpenDictMixin, dict):
    ...     safe(OpenDictMixin.__cache_name__, dict)
    

Classes or Mixins that can be integrated with dict by inheritance must not have a __slots__ definition. Because of that, this mixin must not declare any slot. If needed, it must be declared explicitly in customized classed like in the example in the first part of this documentation or in the definition of opendict class.

class xotl.tools.future.collections.OrderedSmartDict(**kwds)[source]

A combination of the OrderedDict with the SmartDictMixin.

Warning

Initializing with kwargs does not ensure any initial ordering, since Python’s keyword dict is not ordered. Use a list/tuple of pairs instead.

class xotl.tools.future.collections.SmartDictMixin[source]

A mixin that extends the update method of dictionaries

Standard method allow only one positional argument, this allow several.

Note on using mixins in Python: method resolution order is calculated in the order of inheritance, if a mixin is defined to overwrite behavior already existent, use first that classes with it. See SmartDict below.

class xotl.tools.future.collections.StackedDict(**kwargs)[source]

A multi-level mapping.

A level is entered by using the push() and is leaved by calling pop().

The property level returns the actual number of levels.

When accessing keys they are searched from the latest level “upwards”, if such a key does not exists in any level a KeyError is raised.

Deleting a key only works in the current level; if it’s not defined there a KeyError is raised. This means that you can’t delete keys from the upper levels without popping.

Setting the value for key, sets it in the current level.

Changed in version 1.5.2: Based on the newly introduced ChainMap.

pop()[source]

A deprecated alias for pop_level().

Deprecated since version 1.7.0.

push(*args, **kwargs)[source]

A deprecated alias for push_level().

Deprecated since version 1.7.0.

level

Return the current level number.

The first level is 0. Calling push() increases the current level (and returns it), while calling pop() decreases the current level (if possible).

peek()[source]

Peeks the top level of the stack.

Returns a copy of the top-most level without any of the keys from lower levels.

Example:

>>> sdict = StackedDict(a=1, b=2)
>>> sdict.push(c=3)  # it returns the level...
1
>>> sdict.peek()
{'c': 3}
pop_level()[source]

Pops the last pushed level and returns the whole level.

If there are no levels in the stacked dict, a TypeError is raised.

Returns:A dict containing the poped level.
push_level(*args, **kwargs)[source]

Pushes a whole new level to the stacked dict.

Parameters:
  • args – Several mappings from which the new level will be initialled filled.
  • kwargs – Values to fill the new level.
Returns:

The pushed level number.

class xotl.tools.future.collections.ChainMap(*maps)[source]

A ChainMap groups multiple dicts or other mappings together to create a single, updateable view. If no maps are specified, a single empty dictionary is provided so that a new chain always has at least one mapping.

The underlying mappings are stored in a list. That list is public and can accessed or updated using the maps attribute. There is no other state.

Lookups search the underlying mappings successively until a key is found. In contrast, writes, updates, and deletions only operate on the first mapping.

A ChainMap incorporates the underlying mappings by reference. So, if one of the underlying mappings gets updated, those changes will be reflected in ChainMap.

All of the usual dictionary methods are supported. In addition, there is a maps attribute, a method for creating new subcontexts, and a property for accessing all but the first mapping:

maps

A user updateable list of mappings. The list is ordered from first-searched to last-searched. It is the only stored state and can be modified to change which mappings are searched. The list should always contain at least one mapping.

new_child(m=None)[source]

Returns a new ChainMap containing a new map followed by all of the maps in the current instance. If m is specified, it becomes the new map at the front of the list of mappings; if not specified, an empty dict is used, so that a call to d.new_child() is equivalent to: ChainMap({}, *d.maps). This method is used for creating subcontexts that can be updated without altering values in any of the parent mappings.

Changed in version 1.5.5: The optional m parameter was added.

parents

Property returning a new ChainMap containing all of the maps in the current instance except the first one. This is useful for skipping the first map in the search. Use cases are similar to those for the nonlocal keyword used in nested scopes. A reference to d.parents is equivalent to: ChainMap(*d.maps[1:]).

Note

Backported from Python 3.4. In Python 3.4 this is an alias.

class xotl.tools.future.collections.PascalSet(*others)[source]

Collection of unique integer elements (implemented with intervals).

PascalSet(*others) -> new set object

New in version 1.7.1.

class xotl.tools.future.collections.BitPascalSet(*others)[source]

Collection of unique integer elements (implemented with bit-wise sets).

BitPascalSet(*others) -> new bit-set object

New in version 1.7.1.