[ad_1]

Self Supervised Learning (LASSO) is an unsupervised learning method that seeks to discover latent variables or intrinsic structural patterns in datasets [[@B1]]. The original LASSO proposed by Tibshirani and Friedman [[@B3]] is called as regularized LASSO (RLS) which replaces the least absolute deviation (LAD) penalty by a sparse inverse covariance (SIC) penalty, which is a regularization term based on a sparse matrix, to avoid the overfitting problem [[@B2]]. In addition, SIC has one fewer parameter, therefore, has a small bias and is thus more suitable for learning sparse datasets.

In SIC LASSO, it is challenging to accurately determine the sparsity level because the sparsity cannot be directly measured and directly estimated in practice. There were two key challenges in SIC LASSO: the sparsity of the matrix *W*, and the sparsity pattern for the variables. The sparsity of matrix *W*is a necessary condition for the variable selection which ensures the identifiability of the sparse coefficient vector. In practice, the sparsity of the matrix *W*is not straightforward to be measured because it is usually unknown. Therefore, the sparsity pattern of the variables is also a key assumption for the variable selection. However, it is not obvious how to accurately estimate the sparsity pattern of the variables.

Recently, many research studies have revealed the importance of estimating the sparsity pattern of the variables. One of the research directions is to develop methods to identify the intrinsic structure of the variables [[@B4], [@B5]]. In addition, other research directions aim to find an optimal sparsity penalty that makes the matrix *W*to be sparse, by simultaneously optimizing the sparsity penalty and the model parameterization [[@B6], [@B7], [@B8]]. Furthermore, it has been identified that the optimal sparsity penalty must be a solution of a convex optimization problem [[@B9]]. However, the optimal sparsity penalty is not always a solution of a convex optimization problem. This implies that it is not straightforward to find the optimal sparsity penalty. To overcome this situation, there are several existing methods to efficiently estimate the sparsity pattern of the variables. For example, in [[@B10]], the least absolute deviation (LAD) penalty is used to solve the convex optimization problem in a simple way to estimate the sparsity pattern of the variables in an unsupervised learning. In [[@B11]], the sparse inverse covariance (SIC) penalty is adopted to solve the sparse inverse covariance LASSO (SIC-LASSO) problem. These existing methods do not consider that the sparsity pattern of the variables is unknown *a priori*. The existing methods require some initial values to construct the objective function. Such a requirement can be easily satisfied in a case where the variables are known *a priori*. However, the proposed framework does not require initial values. The proposed framework can also be used for an unsupervised learning task using the known sparsity pattern.

The main purpose of our proposed framework is to efficiently solve SIC LASSO. There are several factors that make it difficult to efficiently solve SIC LASSO. First, the sparsity pattern of the variables is unknown a priori. Second, the number of variables is large, many variables have multiple entries, and there are many variables that are highly correlated. Third, the penalty terms are nonlinear functions, and they may not admit an efficient solution. The proposed framework is designed to solve the sparsity pattern of the variables, which can simultaneously determine the sparsity pattern of variables and the penalty terms. For the sparsity pattern of the variables, it can be easily determined and can be estimated based on the sparse coefficient vector. For the penalty terms, it is easy to determine whether they are positive definite or not and whether they are convex or not in the optimization problem.

Moreover, in the SIC LASSO, the problem is considered to be an overcomplete system, in which some of the active variables have no entries. When the number of variables that have multiple entries is small, it is more likely that the active variables have no entries. Compared with the univariate linear regression, the sparse model is overcomplete. The proposed framework can be used for a sparse model and is therefore more suitable for an unsupervised learning task using the known sparsity pattern.

There are two major difficulties in SIC LASSO. First, the sparsity of the matrix *W*is a necessary condition for the variable selection. The sparsity of the matrix *W*is a prerequisite for the variable selection, which ensures the identification of the active variables and the identifiability of the sparse coefficient vector. This is also used to identify the sparsity of the matrix *W.* In the SIC LASSO, it is difficult to accurately determine the sparsity pattern and the sparsity of the matrix *W.* Therefore, the sparsity pattern of the variables is also a key assumption for the variable selection. However, it is not straightforward to determine the sparsity pattern of the variables. In fact, it is not straightforward to determine the sparsity pattern of the variables.

The sparse coefficient vector is the solution of the optimization problem. The proposed framework can be used to solve the sparse coefficient vector. The objective function and the penalization terms are constructed on a basis of a sparse coefficient vector. The sparse coefficient vector is the solution of the optimization problem. Therefore, the proposed framework can be used to solve the sparse coefficient vector.

Thank You.

[ad_2]

Source link