redflag.markov module

Functions related to Markov chains. This code was originally implemented in https://github.com/agilescientific/striplog.

class redflag.markov.Markov_chain(observed_counts, states=None, step=1, include_self=None)

Bases: object

chi_squared(q: float = 0.95) tuple

The chi-squared statistic for the given transition frequencies.

Also returns the critical statistic at the given confidence level q (default 95%).

If the first number is bigger than the second number, then you can reject the hypothesis that the sequence is randomly ordered.

Parameters:

q (float) – The confidence level, as a float in the range 0 to 1. Default: 0.95.

Returns:

The chi-squared statistic.

Return type:

float

property degrees_of_freedom: int
property expected_freqs

The expected frequencies of each state, given the previous state.

classmethod from_sequence(sequence, states=None, strings_are_states=False, include_self=False, step=1)

Parse a sequence and make the transition matrix of the specified order.

You must provide sequence(s) in causal order (e.g. time order).

Parameters:
  • sequence (list-like) – A list-like, or list-like of list-likes. The inner list-likes represent sequences of states. For example, can be a string or list of strings, or a list or list of lists.

  • states (list-like) – A list or array of the names of the states. If not provided, it will be inferred from the data.

  • strings_are_states (bool) –

    True if the strings are themselves states (i.e. words or tokens) and not sequences of one-character states. For example, set to True if you provide something like:

    [‘sst’, ‘mud’, ‘mud’, ‘sst’, ‘lst’, ‘lst’]

  • include_self (bool) – Whether to include self-to-self transitions (default is False: do not include them).

  • step (integer) – The distance to step. Default is 1: use the previous state only. If 2, then the previous-but- one state is used as well as the previous state (and the matrix has one more dimension).

generate_states(n: int = 10, current_state=None)

Generates the next states of the system.

Parameters:
  • n (int) – The number of future states to generate.

  • current_state (str) – The state of the current random variable.

Returns:

list. The next n states.

property normalized_difference

The normalized difference between observed and expected counts.

property observed_freqs

The observed frequencies of each state, given the previous state.

redflag.markov.hollow_matrix(M)

Utility funtion to return hollow matrix (zeros on diagonal).

Args

M (ndarray): a ‘square’ ndarray.

Returns

ndarray. The same array with zeros on the diagonal.

redflag.markov.observations(seq_of_seqs, states, step=1, include_self=False)

Compute observation matrix.

Returns the matrix of transition counts between states.

Parameters:
  • seq_of_seqs (list-like) – A list-like, or list-like of list-likes. The inner list-likes represent sequences of states. For example, can be a string or list of strings, or a list or list of lists.

  • states (list-like) – A list or array of the names of the states. If not provided, it will be inferred from the data.

  • step (integer) – The distance to step. Default is 1: use the previous state only. If 2, then the previous-but- one state is used as well as the previous state (and the matrix has one more dimension).

  • include_self (bool) – Whether to include self-to-self transitions (default is False: do not include them).

Returns:

ndarray. The observation matrix.

redflag.markov.regularize(sequence, strings_are_states=False) tuple

Turn a sequence or sequence of sequences into a tuple of the unique elements in the sequence(s), plus a sequence of sequences (sort of equivalent to np.atleast_2d()).

Args
sequence (list-like): A list-like container of either

states, or of list-likes of states.

strings_are_states (bool): True if the strings are

themselves states (i.e. words or tokens) and not sequences of one-character states. For example, set to True if you provide something like:

[‘sst’, ‘mud’, ‘mud’, ‘sst’, ‘lst’, ‘lst’]

Returns
tuple. A tuple of the unique states, and a sequence

of sequences.