Generic N-fold data partitioner.
Given a dataset with N chunks, with cvtype = 1 (which is default), it would generate N partition sets, where each chunk is sequentially taken out (with replacement) to form a second partition, while all other samples together form the first partition. Example, if there are 4 chunks, partition sets for cvtype = 1 are:
[[1, 2, 3], [0]]
[[0, 2, 3], [1]]
[[0, 1, 3], [2]]
[[0, 1, 2], [3]]
If cvtype>1, then all possible combinations of cvtype number of chunks are taken out, so for cvtype = 2 in previous example yields:
[[2, 3], [0, 1]]
[[1, 3], [0, 2]]
[[1, 2], [0, 3]]
[[0, 3], [1, 2]]
[[0, 2], [1, 3]]
[[0, 1], [2, 3]]
Note that the “taken-out” partition is always labeled ‘2’ while the remaining elements are labeled ‘1’.
If cvtype is a float in the range from 0 to 1, it specifies the ratio of present unique values to be taken.
If cvtype is large enough generating prohibitively large number of combinations, provide count to limit number of combinations and provide selection_strategy = ‘random’.
Notes
Available conditional attributes:
(Conditional attributes enabled by default suffixed with +)
Methods
generate(ds) | |
get_partition_specs(ds) | Returns the specs for all to be generated partition sets. |
get_partitions_attr(ds, specs) | Create a partition attribute array for a particular partion spec. |
get_postproc() | Returns the post-processing node or None. |
get_space() | Query the processing space name of this node. |
reset() | |
set_postproc(node) | Assigns a post-processing node |
set_space(name) | Set the processing space name of this node. |
Parameters: | cvtype : int, float
enable_ca : None or list of str
disable_ca : None or list of str
count : None or int
selection_strategy : str
attr : str
space : str
pass_attr : str, list of str|tuple, optional
postproc : Node instance, optional
descr : str
|
---|
Methods
generate(ds) | |
get_partition_specs(ds) | Returns the specs for all to be generated partition sets. |
get_partitions_attr(ds, specs) | Create a partition attribute array for a particular partion spec. |
get_postproc() | Returns the post-processing node or None. |
get_space() | Query the processing space name of this node. |
reset() | |
set_postproc(node) | Assigns a post-processing node |
set_space(name) | Set the processing space name of this node. |