Measures

Measures

This page contains documentation for all measures found in this package.

JudiLingMeasures.ALC — Method

ALC(cor_s::Union{JudiLing.SparseMatrixCSC, Matrix})

Compute the Average Lexical Correlation (ALC) between the predicted vectors in Shat and all semantic vectors in S.

Arguments

s_cor::Union{JudiLing.SparseMatrixCSC, Matrix}: the correlation matrix between S and Shat

Examples

julia> Shat = [[1 2 1 1]; [1 -2 3 1]; [1 -2 3 3]; [0 0 1 2]]
julia> S = [[-1 2 1 1]; [1 2 3 1]; [1 2 0 1]; [0.5 -2 1.5 0]]
julia> acc, cor_s = JudiLing.eval_SC(Shat, S, R=true)
julia> ALC(cor_s)
4-element Vector{Float64}:
  0.1867546970250672
 -0.030901103469572838
 -0.0681995247218424
  0.011247813283240052

source

JudiLingMeasures.ALDC — Method

ALDC(df::DataFrame)

Compute the Average Levenshtein Distance of all candidates (ALDC) with the correct word form.

Arguments

df::DataFrame: DataFrame of the output of learn_paths.

source

JudiLingMeasures.EDNN — Method

EDNN(Shat::Union{JudiLing.SparseMatrixCSC, Matrix},
     S::Union{JudiLing.SparseMatrixCSC, Matrix})

Compute the Euclidean Distance nearest neighbours between the predicted semantic vectors in Shat and the semantic vectors in Sval and Strain.

source

JudiLingMeasures.EDNN — Method

EDNN(Shat::Union{JudiLing.SparseMatrixCSC, Matrix},
     S::Union{JudiLing.SparseMatrixCSC, Matrix})

Compute the Euclidean Distance nearest neighbours between the predicted semantic vectors in Shat and the semantic vectors in S.

Examples

julia> ma1 = [[1 2 3]; [-1 -2 -3]; [1 2 3]]
julia> ma4 = [[1 2 2]; [1 -2 -3]; [0 2 3]]
julia> EDNN(ma1, ma4)
3-element Vector{Float64}:
 1.0
 2.0
 1.0

source

JudiLingMeasures.L1Norm — Method

L1Norm(M::Union{JudiLing.SparseMatrixCSC, Matrix})

Compute the L1 Norm of each row of a matrix.

Examples

julia> Shat = [[1 2 3]; [-1 -2 -3]; [1 2 3]]
julia> L1Norm(Shat)
3-element Vector{Int64}:
 6
 6
 6

source

JudiLingMeasures.L2Norm — Method

L2Norm(M::Union{JudiLing.SparseMatrixCSC, Matrix})

Compute the L2 Norm of each row of a matrix.

Examples

julia> Shat = [[1 2 3]; [-1 -2 -3]; [1 2 3]]
julia> L2Norm(Shat)
3-element Vector{Float64}:
 3.7416573867739413
 3.7416573867739413
 3.7416573867739413

source

JudiLingMeasures.NNC — Method

NNC(cor_s::Union{JudiLing.SparseMatrixCSC, Matrix})

For each predicted semantic vector get the highest correlation with the semantic vectors in S.

Arguments

s_cor::Union{JudiLing.SparseMatrixCSC, Matrix}: the correlation matrix between S and Shat

Examples

julia> Shat = [[1 2 1 1]; [1 -2 3 1]; [1 -2 3 3]; [0 0 1 2]]
julia> S = [[-1 2 1 1]; [1 2 3 1]; [1 2 0 1]; [0.5 -2 1.5 0]]
julia> acc, cor_s = JudiLing.eval_SC(Shat, S, R=true)
julia> NNC(cor_s)
4-element Vector{Float64}:
 0.8164965809277259
 0.9886230654859615
 0.8625383733289683
 0.35478743759344955

source

JudiLingMeasures.SCPP — Method

SCPP(df::DataFrame, results::DataFrame)

Semantic Correlation of Predicted Production. Returns the correlation of the predicted semantic vector of the predicted path with the target semantic vector.

Arguments

df::DataFrame: The output of learn_paths as DataFrame.
results::DataFrame: The data of interest.

source

JudiLingMeasures.c_precision — Method

c_precision(c_hat_collection, cue_obj)

Calculate the correlation between the predicted and the target cue vector.

Examples

julia> c = [[1. 1. 0.]; [0. 0. 1.]; [1. 0. 1.]]
julia> chat = [[0.9 0.9 0.1]; [0.9 0.1 1.]; [0.9 -0.1 0.8]]
julia> c_precision(chat, c)
3-element Array{Float64,1}:
 1.0
 0.5852057359806527
 0.9958705948858222

source

JudiLingMeasures.density — Method

density(cor_s::Union{JudiLing.SparseMatrixCSC, Matrix};
        n::Int=8, ignore_missing::Bool=false)

Compute the average correlation of each predicted semantic vector with its n most correlated neighbours.

Arguments

s_cor::Union{JudiLing.SparseMatrixCSC, Matrix}: the correlation matrix between S and Shat
n::Int: the number of highest semantic neighbours to take into account

Example

julia> Shat = [[1 2 1 1]; [1 -2 3 1]; [1 -2 3 3]; [0 0 1 2]]
julia> S = [[-1 2 1 1]; [1 2 3 1]; [1 2 0 1]; [0.5 -2 1.5 0]]
julia> acc, cor_s = JudiLing.eval_SC(Shat, S, R=true)
julia> density(cor_s, n=2)
4-element Vector{Float64}:
 0.7393813797301239
 0.6420816485652429
 0.4496869233815781
 0.281150888376636

source

JudiLingMeasures.functional_load — Method

function functional_load(F::Union{JudiLing.SparseMatrixCSC, Matrix},
                         Shat::Union{JudiLing.SparseMatrixCSC, Matrix},
                         cue_obj::JudiLing.Cue_Matrix_Struct;
                         cue_list::Union{Vector{String}, Missing}=missing,
                         method::Union{String, Symbol}="corr")

Correlation/MSE of rows in F of triphones in word w and semantic vector of w.

Measure developed by Motoki Saito. Note: the current version of Functional Load is not completely tested against its original implementation in pyldl.

Arguments

F::Union{JudiLing.SparseMatrixCSC, Matrix}: The comprehension matrix F
Shat::Union{JudiLing.SparseMatrixCSC, Matrix}: The predicted semantic matrix of the data of interest
cueobj::JudiLing.CueMatrix_Struct: The cue object of the data of interest.
cuelist::Union{Vector{String}, Missing}=missing: List of cues for which functional load should be computed. Each cue in the list corresponds to one word in Shat/cueobj and cue and corresponding words have to be in the same order.
method::Union{String, Symbol}="corr": If "corr", correlation between row in F and semantic vector in S is computed. If "mse", mean squared error is used.

Example

julia> using JudiLing, DataFrames, JudiLingMeasures
julia> dat = DataFrame("Word"=>["abc", "bcd", "cde"]);
julia> cue_obj = JudiLing.make_cue_matrix(dat, grams=3, target_col=:Word);
julia> n_features = size(cue_obj.C, 2);
julia> S = JudiLing.make_S_matrix(
    dat,
    ["Word"],
    [],
    ncol=n_features);
julia> F = JudiLing.make_transform_matrix(cue_obj.C, S);
julia> Shat = cue_obj.C * F;
julia> JudiLingMeasures.functional_load(F, Shat, cue_obj)
3-element Vector{Any}:
 [0.9999999999999999, 1.0, 1.0]
 [1.0, 0.9999999999999999, 1.0]
 [0.9999999999999998, 0.9999999999999999, 0.9999999999999998]
julia> JudiLingMeasures.functional_load(F, Shat, cue_obj, cue_list=["#ab", "#bc", "#cd"])
3-element Vector{Any}:
 1.0
 0.9999999999999999
 0.9999999999999998
julia> JudiLingMeasures.functional_load(F, Shat, cue_obj, cue_list=["#ab", "#bc", "#cd"], method="mse")
3-element Vector{Any}:
  8.398316717482945
  8.222104191091363
 14.970231369151817
julia> JudiLingMeasures.functional_load(F, Shat, cue_obj, method="mse")
3-element Vector{Any}:
 [8.398316717482945, 8.398316717482906, 8.398316717482906]
 [8.222104191091363, 8.222104191091224, 8.222104191091226]
 [14.970231369151817, 14.970231369151785, 14.970231369151788]

source

JudiLingMeasures.last_support — Method

last_support(cue_obj::JudiLing.Cue_Matrix_Struct,
             Chat::Union{JudiLing.SparseMatrixCSC, Matrix})

Return the support in Chat for the last ngram of each target word.

source

JudiLingMeasures.lwlr — Method

lwlr(res_learn, pred_df::DataFrame)

The ratio between the predicted form's length and its weakest support from learn_paths.

Arguments

pred_df::DataFrame: The output of get_predicted_path_support

source

JudiLingMeasures.lwlr_chat — Method

lwlr_chat(res_learn, Chat)

The ratio between the predicted form's length and its weakest support in Chat.

source

JudiLingMeasures.mean_word_support — Method

mean_word_support(res_learn, pred_df::DataFrame)

Compute the summed path support divided by each word form's length for each word in dat_val.

Arguments

res_learn: The output of learn_paths
pred_df::DataFrame: The output of get_predicted_path_support

source

JudiLingMeasures.mean_word_support_chat — Method

mean_word_support_chat(res_learn, Chat)

Compute the summed path support, taken from Chat, divided by each word form's length for each word in dat_val.

source

JudiLingMeasures.path_counts — Method

path_counts(df::DataFrame)

Return the number of possible paths as returned by learn_paths.

Arguments

df::DataFrame: DataFrame of the output of learn_paths.

source

JudiLingMeasures.path_entropies_chat — Method

path_entropies_chat(res_learn,
                    Chat::Union{JudiLing.SparseMatrixCSC, Matrix})

source

JudiLingMeasures.path_entropies_scp — Method

path_entropes_scp(df::DataFrame)

Computes the entropy over the semantic supports for all candidates per target word form.

Arguments

df::DataFrame: DataFrame of the output of learn_paths.

source

JudiLingMeasures.path_sum — Method

path_sum(pred_df::DataFrame)

Compute the summed path support for each predicted word with highest support in dat_val.

Arguments

pred_df::DataFrame: The output of get_predicted_path_support

source

JudiLingMeasures.path_sum_chat — Method

path_sum_chat(res_learn,
             Chat::Union{JudiLing.SparseMatrixCSC, Matrix})

source

JudiLingMeasures.rank — Method

rank(cor_s::Union{JudiLing.SparseMatrixCSC, Matrix})

Return the rank of the correct form among the comprehension candidates.

Arguments

s_cor::Union{JudiLing.SparseMatrixCSC, Matrix}: the correlation matrix between S and Shat

Examples

julia> Shat = [[1 2 1 1]; [1 -2 3 1]; [1 -2 3 3]; [0 0 1 2]]
julia> S = [[-1 2 1 1]; [1 2 3 1]; [1 2 0 1]; [0.5 -2 1.5 0]]
julia> acc, cor_s = JudiLing.eval_SC(Shat, S, R=true)
julia> rank(cor_s)
4-element Vector{Any}:
 2
 2
 4
 1

source

JudiLingMeasures.recognition — Method

recognition(data::DataFrame)

Return a vector indicating whether a wordform was correctly understood. Not implemented.

source

JudiLingMeasures.semantic_support_for_form — Method

semantic_support_for_form(cue_obj::JudiLing.Cue_Matrix_Struct,
             Chat::Union{JudiLing.SparseMatrixCSC, Matrix};
             sum_supports::Bool=true)

Return the support in Chat for all target ngrams of each target word.

source

JudiLingMeasures.target_correlation — Method

target_correlation(cor_s::Union{JudiLing.SparseMatrixCSC, Matrix})

Calculate the correlation between each predicted vector and its target vector.

Arguments

Xhat::Union{JudiLing.SparseMatrixCSC, Matrix}: matrix with predicted vectors in rows
X::Union{JudiLing.SparseMatrixCSC, Matrix}: matrix with target vectors in rows

Examples

julia> Shat = [[1 2 1 1]; [1 -2 3 1]; [1 -2 3 3]; [0 0 1 2]]
julia> S = [[-1 2 1 1]; [1 2 3 1]; [1 2 0 1]; [0.5 -2 1.5 0]]
julia> target_correlation(Shat, S)
4-element Vector{Float64}:
  0.6622661785325219
  0.2955402316445243
 -0.86386842558136
  0.35478743759344955

source

JudiLingMeasures.target_correlation — Method

target_correlation(cor_s::Union{JudiLing.SparseMatrixCSC, Matrix})

Calculate the correlation between each predicted semantic vector and its target semantic vector.

Arguments

s_cor::Union{JudiLing.SparseMatrixCSC, Matrix}: the correlation matrix between S and Shat

Examples

julia> Shat = [[1 2 1 1]; [1 -2 3 1]; [1 -2 3 3]; [0 0 1 2]]
julia> S = [[-1 2 1 1]; [1 2 3 1]; [1 2 0 1]; [0.5 -2 1.5 0]]
julia> acc, cor_s = JudiLing.eval_SC(Shat, S, R=true)
julia> target_correlation(cor_s)
4-element Vector{Float64}:
  0.6622661785325219
  0.2955402316445243
 -0.86386842558136
  0.35478743759344955

source

JudiLingMeasures.target_path_sum — Method

target_path_sum(gpi)

Compute the summed path support for each target word. Code by Yu-Ying Chuang.

source

JudiLingMeasures.total_distance — Method

total_distance(cue_obj::JudiLing.Cue_Matrix_Struct,
               FG::Union{JudiLing.SparseMatrixCSC, Matrix},
               mat_type::Symbol)

Code by Yu-Ying Chuang.

source

JudiLingMeasures.uncertainty — Method

function uncertainty(SC_val::Union{JudiLing.SparseMatrixCSC, Matrix},
                     SChat_val::Union{JudiLing.SparseMatrixCSC, Matrix},
                     SC_train::Union{JudiLing.SparseMatrixCSC, Matrix};
                     method::Union{String, Symbol} = "corr")

Sum of correlation/mse/cosine similarity of SChatval with all vectors in SCval and S_train and the ranks of this correlation/mse/cosine similarity.

Measure developed by Motoki Saito. Note: the current version of uncertainty is not completely tested against its original implementation in pyldl.

Arguments

SC::Union{JudiLing.SparseMatrixCSC, Matrix}: S or C matrix of the data of interest
SChat::Union{JudiLing.SparseMatrixCSC, Matrix}: Shat or Chat matrix of the data of interest
method::Union{String, Symbol} = "corr": Method to compute similarity

source

JudiLingMeasures.uncertainty — Method

function uncertainty(SC::Union{JudiLing.SparseMatrixCSC, Matrix},
                     SChat::Union{JudiLing.SparseMatrixCSC, Matrix};
                     method::Union{String, Symbol} = "corr")

Sum of correlation/mse/cosine similarity of SChat with all vectors in SC and the ranks of this correlation/mse/cosine similarity.

Measure developed by Motoki Saito. Note: the current version of uncertainty is not completely tested against its original implementation in pyldl.

Arguments

SC::Union{JudiLing.SparseMatrixCSC, Matrix}: S or C matrix of the data of interest
SChat::Union{JudiLing.SparseMatrixCSC, Matrix}: Shat or Chat matrix of the data of interest
method::Union{String, Symbol} = "corr": Method to compute similarity

Examples

julia> Shat = [[1 2 1 1]; [1 -2 3 1]; [1 -2 3 3]; [0 0 1 2]]
julia> S = [[-1 2 1 1]; [1 2 3 1]; [1 2 0 1]; [0.5 -2 1.5 0]]
julia> JudiLingMeasures.uncertainty(S, Shat, method="corr") # default
4-element Vector{Float64}:
 5.447907056192456
 4.5888162633614
 4.365247579557125
 5.052415166794307
julia> JudiLingMeasures.uncertainty(S, Shat, method="mse")
4-element Vector{Float64}:
 3.5454545454545454
 5.488372093023256
 5.371428571428572
 4.5
julia> JudiLingMeasures.uncertainty(S, Shat, method="cosine")
4-element Vector{Float64}:
 5.749202747845322
 4.308224063773331
 4.423630522948703
 4.877528828745243

source

JudiLingMeasures.within_path_entropies — Method

within_path_entropies(pred_df::DataFrame)

Compute the Shannon Entropy of the path supports for each word in dat_val.

Arguments

pred_df::DataFrame: The output of get_predicted_path_support

source