14 Transition properties modeling

In the subsection 10.0.2, it has been explained that CNMccontrol-oriented Cluster-based Network Modeling has two built-in modal decomposition methods for the \(\boldsymbol Q / \boldsymbol T\) tensors, i .e., SVDSingular Value Decomposition and NMF. There are two main concerns for which performance measurements are needed. First, in subsection 14.0.1, the computational costs of both methods are examined . Then in subsection 14.0.2, the SVDSingular Value Decomposition and NMFNon-negative Matrix Factorization prediction quality will be presented and assessed .

14.0.1 Computational cost

In this subsection, the goal is to evaluate the computational cost of the two decomposition methods implemented in CNMccontrol-oriented Cluster-based Network Modeling. NMFNon-negative Matrix Factorization was already used in first CNMc and it was found to be one of the most computational expensive tasks. With an increasing model order \(L\) it became the most computational task by far, which is acknowledged by (Pierzyna 2021). The run time was one of the main reasons why SVDSingular Value Decomposition should be implemented in CNMccontrol-oriented Cluster-based Network Modeling. To see if SVDSingular Value Decomposition can reduce run time, both methods shall be compared.

First, it is important to mention that NMFNon-negative Matrix Factorization is executed for one single predefined mode number \(r\). It is possible that a selected \(r\) is not optimal, since \(r\) is a parameter that depends not only on the chosen dynamical system but also on other parameters, e.g., the number of centroids \(K\) and training model parameter values \(n_{\beta, tr}\), as well as NMFNon-negative Matrix Factorization specific attributes. These are the maximal number of iterations in which the optimizer can converge and tolerance convergence. However, to find an appropriate \(r\), NMFNon-negative Matrix Factorization can be executed multiple times with different values for \(r\). Comparing the execution time of NMFNon-negative Matrix Factorization with multiple invocations against SVDSingular Value Decomposition can be regarded as an unbalanced comparison. Even though for a new dynamical system and its configuration the optimal \(r_{opt}\) for NMFNon-negative Matrix Factorization is most likely to be found over a parameter study, for the upcoming comparison, the run time of one single NMFNon-negative Matrix Factorization solution is measured.

The model for this purpose is SLS. Since SLS is trained with the output of 7 pairwise different model parameter values \(n_{\beta,tr} = 7\), the maximal rank in SVDSingular Value Decomposition is limited to 7. Nevertheless, allowing NMFNon-negative Matrix Factorization to find a solution \(r\) was defined as \(r=9\), the maximal number of iterations in which the optimizer can converge is 10 million and the convergence tolerance is \(1\mathrm{e}{-6}\). Both methods can work with sparse matrices. However, the SVDSingular Value Decomposition solver is specifically designed to solve sparse matrices. The measured times for decomposing the \(\boldsymbol Q / \boldsymbol T\) tensors for 7 different \(L\) are listed in table 14.1 . It can be observed that for SVDSingular Value Decomposition up to \(L=6\), the computational time for both \(\boldsymbol Q / \boldsymbol T\) tensors is less than 1 second. Such an outcome is efficient for science and industry applications. With \(L=7\) a big jump in time for both \(\boldsymbol Q / \boldsymbol T\) is found. However, even after this increase, the decomposition took around 5 seconds, which still is acceptable.

Table 14.1— Execution time for *SLS* of and for different \(L\)
\(L\)	SVD \(\boldsymbol Q\)	NMF \(\boldsymbol Q\)	SVD \(\boldsymbol T\)	NMF \(\boldsymbol T\)
\(1\)	\(2 \,\mathrm{e}{-4}\) s	\(64\) s	\(8 \, \mathrm{e}{-05}\) s	\(3 \, \mathrm{e}{-2}\) s
\(2\)	\(1 \, \mathrm{e}{-4}\) s	\(8 \, \mathrm{e}{-2}\) s	\(1 \, \mathrm{e}{-4}\) s	\(1\) h
\(3\)	\(2 \, \mathrm{e}{-4}\) s	\(10\) s	\(2 \, \mathrm{e}{-4}\) s	\(0.1\) s
\(4\)	\(4 \, \mathrm{e}{-3}\) s	\(20\) s	\(7 \, \mathrm{e}{-3}\) s	\(1.5\) h
\(5\)	\(6 \, \mathrm{e}{-2}\) s	\(> 3\) h	\(3 \, \mathrm{e}{-2}\) s	-
\(6\)	\(0.4\) s	-	\(0.4\) s	-
\(7\)	\(5.17\) s	-	\(4.52\) s	-

Calculating \(\boldsymbol Q\) with NMFNon-negative Matrix Factorization for \(L=1\) already takes 64 seconds. This is more than SVDSingular Value Decomposition demanded for \(L=7\). The \(\boldsymbol T\) tensor on the other is much faster and is below a second. However, as soon as \(L=2\) is selected, \(\boldsymbol T\) takes 1 full hour, \(L=4\) more than 1 hour. The table for NMFNon-negative Matrix Factorization is not filled, since running \(\boldsymbol Q\) for \(L=5\) was taking more than 3 hours, but still did not finish. Therefore, the time measurement was aborted. This behavior was expected since it was already mentioned in (Pierzyna 2021). Overall, the execution time for NMFNon-negative Matrix Factorization is not following a trend, e.g., computing \(\boldsymbol T\) for \(L=3\) is faster than for \(L=2\) and \(\boldsymbol Q\) for \(L=4\) is faster than for \(L=1\). In other words, there is no obvious rule, on whether even a small \(L\) could lead to hours of run time.

It can be concluded that SVDSingular Value Decomposition is much faster than NMFNon-negative Matrix Factorization and it also shows a clear trend, i.e. the computation time is expected to increase with \(L\). NMFNon-negative Matrix Factorization on the other hand first requires an appropriate mode number \(r\), which most likely demands a parameter study. However, even for a single NMFNon-negative Matrix Factorization solution, it can take hours. With increasing \(L\) the amount of run time is generally expected to increase, even though no clear rule can be defined. Furthermore, it needs to be highlighted that NMFNon-negative Matrix Factorization was tested on a small model, where \(n_{\beta,tr} = 7\). The author of this thesis experienced an additional increase in run time when \(n_{\beta,tr}\) is selected higher. Also, executing NMFNon-negative Matrix Factorization on multiple dynamical systems or model configurations might become infeasible in terms of time. Finally, with the implementation of SVDSingular Value Decomposition, the bottleneck in modeling \(\boldsymbol Q / \boldsymbol T\) could be eliminated.

14.0.2 Prediction quality

In this subsection, the quality of the SVDSingular Value Decomposition and NMFNon-negative Matrix Factorization \(\boldsymbol Q / \boldsymbol T\) predictions are evaluated. The used model configuration for this aim is SLS. First, only the \(\boldsymbol Q\) output with SVDSingular Value Decomposition followed by NMFNon-negative Matrix Factorization shall be analyzed and compared. Then, the same is done for the \(\boldsymbol T\) output.

In order to see how many modes \(r\) were chosen for SVDSingular Value Decomposition the two figures 14.1 and 14.2 are shown. It can be derived that with \(r = 4\), \(99 \%\) of the information content could be captured. The presented results are obtained for \(\boldsymbol Q\) and \(L =1\).

Figure 14.1— *SLS*, , cumulative energy of \(\boldsymbol Q\) for \(L=1\)

Figure 14.2— *SLS*, , singular values of \(\boldsymbol Q\) for \(L=1\)

Figures 14.3 (a) to 14.3 (c) depict the original \(\boldsymbol{Q}(\beta_{unseen} = 28.5)\), which is generated with CNM, the CNMccontrol-oriented Cluster-based Network Modeling predicted \(\boldsymbol{\tilde{Q}}(\beta_{unseen} = 28.5)\) and their deviation \(| \boldsymbol{Q}(\beta_{unseen} = 28.5) - \boldsymbol{\tilde{Q}}(\beta_{unseen} = 28.5) |\), respectively. In the graphs, the probabilities to move from centroid \(c_p\) to \(c_j\) are indicated. Contrasting figure 14.3 (a) and 14.3 (b) exhibits barely noticeable differences. For highlighting present deviations, the direct comparison between the CNMCluster-based Network Modeling and CNMccontrol-oriented Cluster-based Network Modeling predicted \(\boldsymbol Q\) tensors is given in figure 14.3 (c) . It can be observed that the highest value is \(max( \boldsymbol{Q}(\beta_{unseen} = 28.5) - \boldsymbol{\tilde{Q}}(\beta_{unseen} = 28.5) |) \approx 0.0697 \approx 0.07\). Note that all predicted \(\boldsymbol Q\) and \(\boldsymbol T\) tensors are obtained with RFRandom Forest as the regression model.

(a) Original \(\boldsymbol{Q}(\beta_{unseen} = 28.5)\)

The same procedure shall now be performed with NMF. The results are depicted in figures 14.4 and 14.5 . Note that the original CNMCluster-based Network Modeling \(\boldsymbol{Q}(\beta_{unseen} = 28.5)\) does not change, thus figure 14.3 (a) can be reused. By exploiting figure 14.6, it can be observed that the highest deviation for the NMFNon-negative Matrix Factorization version is \(max( \boldsymbol{Q}(\beta_{unseen} = 28.5) - \boldsymbol{\tilde{Q}}(\beta_{unseen} = 28.5) |) \approx 0.0699 \approx 0.07\). The maximal error of NMFNon-negative Matrix Factorization \((\approx 0.0699)\) is slightly higher than that of SVDSingular Value Decomposition \((\approx 0.0697)\). Nevertheless, both methods have a very similar maximal error and seeing visually other significant differences is hard.

Figure 14.4— predicted \(\boldsymbol{\tilde{Q}}(\beta_{unseen} = 28.5)\)

Figure 14.5— Deviation \(| \boldsymbol{Q}(\beta_{unseen} ) - \boldsymbol{\tilde{Q}}(\beta_{unseen} ) |\)

SLS, NMFNon-negative Matrix Factorization, CNMccontrol-oriented Cluster-based Network Modeling predicted \(\boldsymbol{\tilde{Q}}(\beta_{unseen} = 28.5)\) and deviation \(| \boldsymbol{Q}(\beta_{unseen} = 28.5) - \boldsymbol{\tilde{Q}}(\beta_{unseen} = 28.5) |\) for \(L=1\)

In order to have a quantifiable error value, the Mean absolute error (MAE) following equation 10.4 is leveraged. The MAE errors for SVDSingular Value Decomposition and NMFNon-negative Matrix Factorization are \(MAE_{SVD} = 0.002 580 628\) and \(MAE_{NMF} = 0.002 490 048\), respectively. NMFNon-negative Matrix Factorization is slightly better than SVDSingular Value Decomposition with \(MAE_{SVD} - MAE_{NMF} \approx 1 \mathrm{e}{-4}\), which can be considered to be negligibly small. Furthermore, it must be stated that SVDSingular Value Decomposition was only allowed to use \(r_{SVD} = 4\) modes, due to the \(99 \%\) energy demand, whereas NMFNon-negative Matrix Factorization used \(r_{NMF} = 9\) modes. Given that SVDSingular Value Decomposition is stable in computational time, i.e., it is not assumed that for low \(L\), the computational cost scales up to hours, SVDSingular Value Decomposition is the clear winner for this single comparison.

For the sake of completeness, the procedure shall be conducted once as well for the \(\boldsymbol T\) tensor. For this purpose figures 14.6 to 14.10 shall be considered. It can be inspected that the maximal errors for SVDSingular Value Decomposition and NMFNon-negative Matrix Factorization are \(max( \boldsymbol{T}(\beta_{unseen} = 28.5) - \boldsymbol{\tilde{T}}(\beta_{unseen} = 28.5) |) \approx 0.126\) and \(max( \boldsymbol{T}(\beta_{unseen} = 28.5) - \boldsymbol{\tilde{T}}(\beta_{unseen} = 28.5) | ) \approx 0.115\), respectively. The MAE errors are, \(MAE_{SVD} = 0.002 275 379\) and \(MAE_{NMF} = 0.001 635 510\). NMFNon-negative Matrix Factorization is again slightly better than SVDSingular Value Decomposition with \(MAE_{SVD} - MAE_{NMF} \approx 6 \mathrm{e}{-4}\), which is a deviation of \(\approx 0.06 \%\) and might also be considered as negligibly small.

Figure 14.6— Original \(\boldsymbol{T}(\beta_{unseen} = 28.5)\)

Figure 14.7— predicted \(\boldsymbol{\tilde{T}}(\beta_{unseen} = 28.5)\)

Figure 14.8— Deviation \(| \boldsymbol{T}(\beta_{unseen}) - \boldsymbol{\tilde{T}}(\beta_{unseen}) |\)

SLS, SVDSingular Value Decomposition, original \(\boldsymbol{T}(\beta_{unseen} = 28.5)\), predicted \(\boldsymbol{\tilde{T}}(\beta_{unseen} = 28.5)\) and deviation \(| \boldsymbol{T}(\beta_{unseen} = 28.5) - \boldsymbol{\tilde{T}}(\beta_{unseen} = 28.5) |\) for \(L=1\)

Figure 14.9— predicted \(\boldsymbol{\tilde{T}}(\beta_{unseen} = 28.5)\)

Figure 14.10— Deviation \(| \boldsymbol{T}(\beta_{unseen}) - \boldsymbol{\tilde{T}}(\beta_{unseen}) |\)

SLS, NMFNon-negative Matrix Factorization, CNMccontrol-oriented Cluster-based Network Modeling predicted \(\boldsymbol{\tilde{T}}(\beta_{unseen} = 28.5)\) and deviation \(| \boldsymbol{T}(\beta_{unseen} = 28.5) - \boldsymbol{\tilde{T}}(\beta_{unseen} = 28.5) |\) for \(L=1\)

Additional MAE errors for different \(L\) and \(\beta_{unseen}= 28.5,\, \beta_{unseen}= 32.5\) are collected in table 14.2 . First, it can be outlined that regardless of the chosen method, SVDSingular Value Decomposition or NMFNon-negative Matrix Factorization, all encountered MAE errors are very small. Consequently, it can be recorded that CNMccontrol-oriented Cluster-based Network Modeling convinces with an overall well approximation of the \(\boldsymbol Q / \boldsymbol T\) tensors. Second, comparing SVDSingular Value Decomposition and NMFNon-negative Matrix Factorization through their respective MAE errors, it can be inspected that the deviation of both is mostly in the order of \(\mathcal{O} \approx 1 \mathrm{e}{-2}\). It is a difference in \(\approx 0.1 \%\) and can again be considered to be insignificantly small.

Despite this, NMFNon-negative Matrix Factorization required the additional change given in equation 14.1, which did not apply to SVDSingular Value Decomposition. The transition time entries at the indexes where the probability is positive should be positive as well. Yet, this is not always the case when NMFNon-negative Matrix Factorization is executed. To correct that, these probability entries are manually set to zero. This rule was also actively applied to the results presented above. Still, the outcome is very satisfactory, because the modeling errors are found to be small.

Table 14.2— *SLS*, Mean absolute error for different \(L\) and two \(\beta_{unseen}\)
\(L\)	\(\beta_{unseen}\)	\(\boldsymbol{MAE}_{SVD, \boldsymbol Q}\)	\(\boldsymbol{MAE}_{NMF, \boldsymbol Q}\)	\(\boldsymbol{MAE}_{SVD, \boldsymbol T}\)	\(\boldsymbol{MAE}_{NMF, \boldsymbol T}\)
\(1\)	\(28.5\)	\(0.002580628\)	\(0.002490048\)	\(0.002275379\)	\(0.001635510\)
\(1\)	\(32.5\)	\(0.003544923\)	\(0.003650155\)	\(0.011152145\)	\(0.010690052\)
\(2\)	\(28.5\)	\(0.001823848\)	\(0.001776276\)	\(0.000409955\)	\(0.000371242\)
\(2\)	\(32.5\)	\(0.006381635\)	\(0.006053059\)	\(0.002417142\)	\(0.002368680\)
\(3\)	\(28.5\)	\(0.000369228\)	\(0.000356817\)	\(0.000067680\)	\(0.000062964\)
\(3\)	\(32.5\)	\(0.001462458\)	\(0.001432738\)	\(0.000346298\)	\(0.000343520\)
\(4\)	\(28.5\)	\(0.000055002\)	\(0.000052682\)	\(0.000009420\)	\(0.000008790\)
\(4\)	\(32.5\)	\(0.000215147\)	\(0.000212329\)	\(0.000044509\)	\(0.000044225\)

\[ \begin{equation} \begin{aligned} TGZ := \boldsymbol T ( \boldsymbol Q > 0) \leq 0 \\ \boldsymbol Q ( TGZ) := 0 \end{aligned} \label{eq_33} \end{equation} \tag{14.1}\]

In summary, both methods NMFNon-negative Matrix Factorization and SVDSingular Value Decomposition provide a good approximation of the \(\boldsymbol Q / \boldsymbol T\) tensors. The deviation between the prediction quality of both is negligibly small. However, since SVDSingular Value Decomposition is much faster than NMFNon-negative Matrix Factorization and does not require an additional parameter study, the recommended decomposition method is SVDSingular Value Decomposition. Furthermore, it shall be highlighted that SVDSingular Value Decomposition used only \(r = 4\) modes for the \(\boldsymbol Q\) case, whereas for NMFNon-negative Matrix Factorization \(r=9\) were used. Finally, as a side remark, all the displayed figures and the MAE errors are generated and calculated with CNMCluster-based Network Modeling’s default implemented methods.