quara.math.entropy module

gradient_relative_entropy_2nd(prob_dist_q, prob_dist_p, gradient_prob_dist_ps, eps_q=None, eps_p=None, is_valid_required=True, atol=None)

returns gradient of relative entropy of probability distributions q and p.

Parameters

• prob_dist_q (np.ndarray) – a probability distribution q.

• prob_dist_p (np.ndarray) – a probability distribution p.

• gradient_prob_dist_ps (np.ndarray) – gradients of probability distribution p. ndim of this parameter must be 2 (list of gradients).

• eps_q (float, optional) – a parameter to avoid divergence about q, by default 1e-10

• eps_p (float, optional) – a parameter to avoid divergence about p, by default 1e-10

• is_valid_required (bool, optional) – if is_valid_required is True, then check whetever the entries of prob_dist_p is a negative number, uses True by default.

• atol (float, optional) – the absolute tolerance parameter, uses get_atol() by default.

Returns

gradient of relative entropy of probability distributions q and p.

Return type

np.ndarray

gradient_relative_entropy_2nd_vector(prob_dist_q, prob_dist_p, gradient_prob_dist_ps, eps_q=None, eps_p=None, is_valid_required=True, atol=None)

returns pointwise gradient of relative entropy of probability distributions q and p.

Parameters

• prob_dist_q (np.ndarray) – a probability distribution q.

• prob_dist_p (np.ndarray) – a probability distribution p.

• gradient_prob_dist_ps (np.ndarray) – gradients of probability distribution p. ndim of this parameter must be 2 (list of gradients).

• eps_q (float, optional) – a parameter to avoid divergence about q, by default 1e-10

• eps_p (float, optional) – a parameter to avoid divergence about p, by default 1e-10

• is_valid_required (bool, optional) – if is_valid_required is True, then check whetever the entries of prob_dist_p is a negative number, uses True by default.

• atol (float, optional) – the absolute tolerance parameter, uses get_atol() by default.

Returns

gradient of relative entropy of probability distributions q and p.

Return type

np.ndarray

hessian_relative_entropy_2nd(prob_dist_q, prob_dist_p, gradient_prob_dist_ps, hessian_prob_dist_ps, eps_q=None, eps_p=None, is_valid_required=True, atol=None)

returns Hessian of relative entropy of probability distributions q and p.

Parameters

• prob_dist_q (np.ndarray) – [description]

• prob_dist_p (np.ndarray) – [description]

• gradient_prob_dist_ps (np.ndarray) – gradients of probability distribution p. ndim of this parameter must be 2 (list of gradients).

• hessian_prob_dist_ps (np.ndarray) – Hessians of probability distribution p. ndim of this parameter must be 3 (list of Hessians).

• eps_q (float, optional) – a parameter to avoid divergence about q, by default 1e-10

• eps_p (float, optional) – a parameter to avoid divergence about p, by default 1e-10

• is_valid_required (bool, optional) – if is_valid_required is True, then check whetever the entries of prob_dist_p is a negative number, uses True by default.

• atol (float, optional) – the absolute tolerance parameter, uses get_atol() by default.

Returns

Hessian of relative entropy of probability distributions q and p.

Return type

float

relative_entropy(prob_dist_q, prob_dist_p, eps_q=None, eps_p=None, is_valid_required=True, atol=None)

returns relative entropy of probability distributions q and p.

Parameters

• prob_dist_q (np.ndarray) – a probability distribution q.

• prob_dist_p (np.ndarray) – a probability distribution p.

• eps_q (float, optional) – a parameter to avoid divergence about q, by default 1e-10

• eps_p (float, optional) – a parameter to avoid divergence about p, by default 1e-10

• is_valid_required (bool, optional) – if is_valid_required is True, then check whetever the entries of prob_dist_p is a negative number, uses True by default.

• atol (float, optional) – the absolute tolerance parameter, uses get_atol() by default.

Returns

relative entropy of probability distributions q and p.

Return type

float

relative_entropy_vector(prob_dist_q, prob_dist_p, eps_q=None, eps_p=None, is_valid_required=True, atol=None)

returns pointwise relative entropy of probability distributions q and p.

Parameters

• prob_dist_q (np.ndarray) – a probability distribution q.

• prob_dist_p (np.ndarray) – a probability distribution p.

• eps_q (float, optional) – a parameter to avoid divergence about q, by default 1e-10

• eps_p (float, optional) – a parameter to avoid divergence about p, by default 1e-10

• is_valid_required (bool, optional) – if is_valid_required is True, then check whetever the entries of prob_dist_p is a negative number, uses True by default.

• atol (float, optional) – the absolute tolerance parameter, uses get_atol() by default.

Returns

relative entropy of probability distributions q and p.

Return type

float

round_varz(z, eps, is_valid_required=True, atol=None)

returns max{z , eps}.

This function to be used to avoid divergence. Both the arguments z and eps must be negative real numbers.

Parameters

• z (Union[float, np.float64]) – variable z.

• eps (Union[float, np.float64]) – variable eps.

• is_valid_required (bool, optional) – if is_valid_required is True, then check whetever z is a negative number, uses True by default.

• atol (float, optional) – the absolute tolerance parameter, uses get_atol() by default.

Returns

max{z , eps}.

Return type

np.float64

Raises

• ValueError – z is not a real number(float or np.float64).

• ValueError – z is a negative number.

• ValueError – eps is not a real number(float or np.float64).

• ValueError – eps is a negative number.

round_varz_vector(z, eps, is_valid_required=True, atol=None)

returns pointwise max{z , eps}.

This function to be used to avoid divergence. Both the arguments z and eps must be negative real numbers.

Parameters

• z (Union[float, np.float64]) – variable z.

• eps (Union[float, np.float64]) – variable eps.

• is_valid_required (bool, optional) – if is_valid_required is True, then check whetever z is a negative number, uses True by default.

• atol (float, optional) – the absolute tolerance parameter, uses get_atol() by default.

Returns

max{z , eps}.

Return type

np.float64

Raises

• ValueError – z is not a real number(float or np.float64).

• ValueError – z is a negative number.

• ValueError – eps is not a real number(float or np.float64).

• ValueError – eps is a negative number.