An Adaptive Approach to Optimize Probabilistic Distributed Testing

Typically, conformance testing consists of placing a set of parallel testers at each port of an implementation to ensure its conformance to the specification. However, a number of common fault detections occur if no coordination is made between these parallel testers and the implementation under test (IUT). Therefore, the test process must support mechanisms of coordination between these distributed components, particularly for implementation with stochastic behaviour. To this end, as well as to analyse the stochastic behaviour of the implementation under test, we propose in this paper an algorithm to generate for each tester a probabilistic local test sequence (PLTS) aiming to avoid both synchronization and observation issues. Finally, we suggest a new architecture based on Markov decision processes with an adaptive controller to control and optimize the whole testing process.


Introduction
The latest developments in the field of distributed systems have led to an emphasis on testing aspects of these systems. Hence, an area referred by "Model Based Testing" (MBT) has emerged in which systems' testing is based either on models or on specifications. In the context of distributed testing, we usually consider the implementation under test (IUT) as a "black box" with some distributed interfaces, called ports and the behaviour of the IUT is known only by its interactions through these ports. In addition, we place a tester at each physical interface of the implementation (IUT) to perform our tests. However, the challenge lies in ensuring coordination between such testers and the implementation under test. In fact, several aspects such as the fault detections for example, could be significantly influenced by lack of coordination.
Overall, we face three main problems when testing distributed systems. These problems are known as controllability, observability and synchronization fault detections which are fundamental features of conformance distributed testing. These problems have a great influence on testing process of distributed testing systems, such as the execution of test sequences, the fault detectability of test system and the interpretation of testing results [1,2,3]. As solution to these problems, many works propose that testers exchange some messages through reliable communication channels [4] or to use eventually some timing constraints as explained in [5,6].
On the other hand, previous works done on the distributed testing field have not widely addressed probabilistic models, even though distributed systems have often probabilistic features and characteristics. To this end, we use Probabilistic Automata (PAs) to define the probabilities to be associated with the events and particularly, we examine the properties used to specify the probability of exchanging messages among the various components of 2. Testing distributed systems

Architecture
The principle of the distributed test architecture is to coordinate a set of parallel test components (testers) with the implementation under test (IUT) using a communication service (Fig.1).

Test System
Multicast Channel In fact, the communication between testers and the IUT is performed through a set of ports called PCO (Port of Control and Observation). However, the testers communicate with each other using a multicast channel.

Distributed Testing problems
There are a number of issues that influence fault detection during the conformance testing process. In the main, three major problems known as controllability, observability and synchronization fault detections arise if no coordination ensured between the distributed test components: • Controllability problem: refers to the test system ability to handle events at the corresponding ports PCOs in a given order. Indeed, the controllability problem arises while the Test system cannot guarantee that the implementation receives event of transition (i) before event of transition (i+1).
• Observability problem: It can be defined from Test System view as the capability of a Test System to observe the outputs of the IUT and decide which input is the cause of each output, it arises when two consecutive transition (i) and transition (i+1) occur on the same port k but only one of the transitions has an output in port k and the other one is an empty transition with no output. In this case the Test System cannot decide whether transition (i) or transition (i+1) is the cause of the output.
• Synchronization problem: In the distributed test, the testers will interact with the IUT through the communication channel. In order to reason about the synchronization problem and discuss the causality between the observed events, it is necessary to make assumptions regarding message latency.
In this paper, we aim to extend the distributed testing model on a probabilistic model by specifying the probabilistic behaviour. To this end, we review first some related works in the literature.

Literature review
Many works studied probabilistic testing of distributed systems. In this context, the authors in [7] proposed a framework to test formally systems with distributed ports where some choices are probabilistically quantified while other choices are non-deterministic. The main idea in [1,2] is to construct a test sequence that causes no controllability or observability problems during its application in distributed test architecture. For some specifications, such sequence exists where the coordination is achieved via interactions with the IUT [8]. However, this case is not always true as detailed in [1,9].
The authors in [10] suggest using multiple unique input/output (UIO) sequences to generate a test sequence and a method to minimize input/output operations as well as external coordination messages. Moreover, two model-based testing frameworks are explained in [11]. They additionally cover the stochastic aspects in hard and soft real-time systems. The authors use the stochastic automata as well as the theory of Markov automata for specifications, test cases, and conformance. Furthermore, the authors in [12] implement a new distribute cloud testing architecture for conformance testing to overcome problems occurred when testing distributed cloud. Thereafter, they extend their study to present in [13] a novel secure architecture-based conformance testing in order to check not only if the Cloud environment is conforming to its specification but also if respects some security policies using distributed testing rules.
The work [14] introduces the fundamental concepts of probabilistic bi-simulation via hypothesis testing whereas the authors in [15] show how to observe trace probabilities via hypothesis testing. Furthermore, the authors developed in [16] mutation testing techniques for probabilistic finite state machines (PFSMs) and probabilistic-stochastic finite state machines (PSFSMs), to produce test sequences that distinguish a PFSM M from a mutant M'. In addition, the aim of the statistical testing is to select entries for an IUT implementation in a probabilistic way in order to optimise some test measures as described in [17,18,19,20]. Furthermore, an executable model-based testing framework for probabilistic systems is provided in [21]. The authors present therefore algorithms to automatically execute and assess test cases that are generated from a probabilistic specification.
Finally, this paper deals with the probabilistic properties of distributed testing and can be considered as a continuity of previous works [22,23,24]. In fact, the paper enhances our algorithms described in [25,26,27] for the generation of local test sequence. More precisely, the paper focus on the constraints that specify the probability of exchanging messages between testers as well as optimizing testing in distributed systems using an adaptive controller.

Preliminaries
In this section, we extend the definition of a probabilistic automaton with n-port to formally model and specify the stochastic behaviour of the distributed testing system.
For this purpose, we give some definitions of the n-ports probabilistic automata in distributed testing context as follows: Definition1. We define a n-ports Probabilistic Automaton (np-PA) by A = (S, s 0 , Act, T) with: • S is a finite set of states; • s 0 ∈ S is the initial state; 150 AN ADAPTIVE APPROACH TO OPTIMIZE PROBABILISTIC DISTRIBUTED TESTING • Act = Σ ∪ Γ , is a finite set of actions with: Definition2. We define a transition as a tuple (s 1 , σ, s 2 , p[0, 1]) where: • s 1 , s 2 ∈ S are the source state and destination state; • σ ∈ Act, is a sending of an input "x i " or reception of an output "y i " (figured respectively as !x i and ?y j ); • p[0,1]: For each T ∈ Tr, we associate a probability p. Therefore, p is a non-negative real number 1 and the sum of the probabilities leaving a state: s ∈ S is 1.
The meaning of T (s 1 , σ, s 2 , p) is that we can move from state s 1 to state s 2 with action σ ∈ Act with probability p ∈ [0,1].

PGTS: Probabilistic global test sequence
We define a probabilistic global test sequence (PGTS) as a test sequence of an np-PA corresponding to the sequence of transitions: Tr !x1 .Tr ?y1 .Tr !x2 .Tr ?y2 ...Tr !xt .Tr ?yt where: • Tr !xi is a transition of sending an input "x i " in port-k. We denote: <!x i , Prob(i)> where Prob(i) corresponds to the probability of sending an input; • Tr ?yi are the outputs received on different ports (1, 2, ..., j) with 1 j n resulting on sending an input "xi" to the IUT. We denote: The transitions corresponding to the example above (Fig.2) are expressed as follows: • Tr !a1 = (!a 1 , 1); Subsequently, we get an example of PGTS of 3p-PA given in Fig.2 as:

PLTS: Probabilistic local test sequences
Generation of Probabilistic Local Test Sequences (PLTS): We suggest in this section our algorithm aiming to build the PLTS sequences related to each tester from a complete probabilistic global test sequence (PGTS) in order to ensure coordination and avoid observability and synchronisation issues.
A PLTS sequence has the form Tr 1 .Tr 2 .Tr 3 ....Tr n where each Tr i is either: it is a message exchanged between tester-k and IUT. Each M i is: • !x i : Tester-k sends "x i " through port k to IUT or; • ?y i : IUT sends "y i " through port k to tester-k. 2. M k i if it is a synchronization or observation message from a tester-k. Each M i is: Observation message (sent to/received from) tester-k or; • !S k /?S k : Synchronization message S (sent to/received from) tester-k.
Therefore, each transition is considered as a data structure that contains the message to be exchanged as well as the probability of the event. We denote:   By taking the global probabilistic test sequence defined in (1) as an example, the algorithm 1 above generates for each tester a local probabilistic test sequence (PLTS) as follows: • Basically, the PLTS sequences are projections of the global probabilistic test sequence over the port alphabets; • We add the reception of messages belonging to "y i " to the appropriate sequences: The loop in (line 16); • We add "?O k " (k is port sending O) and "!O k " (k is port receiving O) to the appropriate local test sequences (lines 4 to 14) in order to avoid observation problems. In fact, each tester receiving a message h ∈ y i−1 should be able to determinate that h has been sent by IUT after IUT has received "x i−1 " and before IUT receives "x i ". • We remind that the synchronization problem arises when we cannot guarantee that IUT will receive event of transition (i) before event of transition (i+1). To avoid Synchronization problem, we added "?S k " (k is port sending S) and "!S k " (k is port receiving S) to the appropriate local test sequences as follows: -"?S k " is added to w ph where h is the tester sending "x i+1 " (lines 25 and 30); -"!S k " is added to the sequence of a tester receiving a message belonging to "y i ", if y i = ∅ (lines 26 to 32); if not "!S k " is added to the sequence of the tester sending "x i " (lines 23 to 25).
As a result, we get the following PLTS sequences by applying our algorithm 1 to the PGTS giving in (1). These PLTS sequences (W P 1 , W P 2 and W P 3 ) describe the behaviour of tester-1, tester-2 and tester-3 respectively: Afterwards, each tester will execute the following procedure during the execution of its probabilistic local test sequence in order to get the test verdict: • For each message "x i " sending to the IUT or a synchronization/observation message, the tester supports the process of sending this message; • If "M i " is an expected message from the IUT or a synchronization/observation message, the tester waits for this message. If no message is received, or if the received message is not expected, the tester returns a verdict Fail (fail); • If the tester reaches the end of its local test sequence, then it gives a verdict Accept (accepted). Thus, if all testers return a verdict Accept, then the test system ends the test with a global verdict Accept. Otherwise, the test system ends with Fail verdict.
In the next section, we introduce the Markov Decision Process to model the stochastic behaviour of the distributed testing system and to optimize the Test execution.

Optimal distributed test control architecture
The figure (Fig. 3)   To model the stochastic behaviour of our approach, we propose a formal model based on the MDP definition (Markov Decision Processes). Therefore, we extend some definitions related to the MDPs for inputs and outputs in distributed testing mode.

Markov decision processes: Definitions
Definition 3. A MDP is a 4-tuples (S, A, T, R) where: • S is a finite set of states. • s 0 ∈ S is the initial state. • A = Σ ∪ Γ is a finite set of actions, with: i. Σ = {Σ 1 , Σ 2 , ..., Σ n } where Σ i is a finite set of inputs of ports i, Σ i ∩ Σ j = ∅ for i = j and i,j = 1,2,...,n and Σ = Σ 1 ∪ Σ 2 ∪ ... ∪ Σ n ; ii. Γ = {Γ 1 , Γ 2 , ..., Γ n } where Γ i is a finite set of outputs of ports i, Γ i ∩ Γ j = ∅ for i = j and i,j = 1,2,...,n and Γ = Γ 1 ∪ Γ 2 ∪ ... ∪ Γ n ; • T : S × A × S ∈ [0, 1] is a transition model. T(s, a, s') is the probability that if a controller performs action a∈ A when in state: s ∈ S. Then, the state changes into state: s' ∈ S. T is constrained such that the sum of the probabilities leaving a state: s ∈ S with action: a ∈ A is 1; • R : S × A × S → R, is the immediate reward function, where R(s, a, s') is the immediate reward received after transitioning from state: s ∈ S to state: s' ∈ S, due to action a ∈ A.
Definition 4. A policy π: S → A is a function that specifies the action π(s) ∈ A that the controller will choose when in state: s ∈ S and π * denotes the optimal policy that optimizes the expected rewards.
Definition 5. A state value function V: S → R induced by π is denoted V π (s). V π (s) is called the state value function for a given policy π. the value V π (s) is the expected cumulative reward that a controller will receive if it follows the policy π when starting from state: s ∈ S. We compute V π (s) for each state: s ∈ S as follows: V π (s) = R(s, π(s), s ) + γ s ∈S T (s, π(s), s )V π (s ) Where: γ ∈ [0, 1] is the reward discount factor. Definition 6. A state-action value function Q: S × A → R induced for a given policy π is denoted Q π (s, a). The value Q π (s, a) is the expected cumulative reward that a controller will receive when starting in state: s ∈ S, and executing the action a ∈ A, then following the policy π. We compute Q π (s, a) for each state s ∈ S as: Q π (s, a) = R(s, a, s ) + γ s ∈S T (s, a, s )V π (s ) Definition 7. In a Markov decision process, an optimal policy π * is the policy that maximizes the expected reward, and satisfying: According to the Bellman's equation [28], the corresponding optimal value function (denoted V * = V π * ) is defined as the best value that can be achieved in a state: s ∈ S, where:

Optimal control test process
The basic idea is to use a controller that will learn the optimal policy based on the specification of the (IUT) implementation.

Case Study
In this section, we propose a Markov Decision Processes (MDP) example to illustrate the stochastic behaviour of the distributed testing system. Therefore, we will use the MDP to model the synchronization and observability states that distributed testing system should avoid as well as rewards that the controller of the distributed testing system will apply in a given state if needed.
The figure (Fig. 4) gives an example of a MDP model of a 3p-IUT. The example is defined with the following parameters: • S = {s 0 , s 1 , ..., s 9 } a finite set of states; • s 0 ∈ S the initial state; such as:

Optimal control test Execution
In order to test the conformity of the IUT, the optimal control Test process is composed of two phases: • The learning phase where the controller learns from the model based of the specification, to act optimally.
• The testing phase where the controller controls the distributed test based on the optimal policy learned in the first phase.
Therefore, the adaptive controller will learn the optimal policy from the specification, and based on the learning process the controller will make the decision to send an input "!x i " to the IUT, or to wait for an output "?y i " from the IUT.

Learning algorithm
In the learning process, the adaptive controller will apply the algorithm below to map the optimal policy based on the MDP model of the specification.
Afterwards, the adaptive controller will use the computed optimal policies to optimize and control the distributed testing process. In fact, the algorithm generates a deterministic policy from the model of the specification as follows: • The algorithm takes as input the transition model T, the immediate reward function R, the discount factor γ, and the accuracy of estimation . • We initialize the states values to 0 (lines 1 to 3) and we update then values for each state s ∈ S by applying the Bellman equation to the function value iteratively (lines 6 to 10) until we converge to the optimal value (lines 4, 5, 11). • We get as output a deterministic policy π that is the greedy policy with respect of the optimal value function for every state s ∈ S.

Algorithm 2 Learning Optimal Policy.
Inputs T is the transition model; R is the immediate reward function; γ the discount factor; the accuracy of estimation. After generating an optimal policy, the adaptive controller will learn to act optimally and choose the optimal action at each state in order to avoid observability and synchronization problems during distributed testing process.
The interaction between the IUT and the distributed testing system is considered asa decision making problem where the adaptive controller chooses actions based on the observed behaviour of the IUT and the optimal policy learned using the system specification.
An example of an optimal global control test sequence (OGCTS) deduced from the MDP model given in the figure (Fig.4)  And the local test sequences at each port is given as follows: Therefore, the distributed testers will not need to exchange any kind of coordination messages as the adaptive controller will guarantee avoiding observability, and synchronization problems based on the optimal learning phase.

Test procedure
During the execution of the OGCTS, the adaptive controller will execute the following procedure: • At each stage of the distributed testing process, the adaptive controller will choose the action to take based on the state of the IUT: -If it is a sending message to the IUT, the adaptive controller makes the corresponding tester supports the process of sending this message; -If it is an expected message from the IUT, the adaptive controller waits for this message from testers.
if no message is received, or if the received message is not expected, the adaptive controller returns a verdict Fail.
• if the adaptive controller reaches the end of its OGCTS, then it gives a verdict Accept. Then the test system ends the test with a verdict Accept.

Discussion and Concluding Remarks
As a conclusion, we intended in this paper to use the core of probabilistic test theory to analyse the behaviour of the implementation under test. For this purpose, the paper focuses on the constraints that specify the probability of exchanging messages between testers as well as the mechanisms needed to optimize the distributed testing process. In this context, we propose an algorithm to generate probabilistic local test sequences (PLTS)in order to ensure coordination between testers without encountering observation and synchronization problems. Compared to other previous studies that propose a model without a multicast channel between testers such as [7].The authors define indeed a weak implementation relationship based on probabilistic input and output transition systems. Our model, on the other hand, uses this channel to communicate messages between testers and subsequently ensure communication between testers to address the synchronization problem between testers. Moreover, the algorithm we proposed minimizes the messages exchanged between the testers, using only the observability and synchronization messages when needed. To achieve this, we used probabilistic automata to define the probabilities and model the messages exchanged between local testers and the IUT, allowing us to generate a probabilistic local test sequence (PLTS) for each tester by guaranteeing the absence of coordination, observability, and synchronization problems.
Furthermore, we suggest a new architecture based on Markov decision processes with an adaptive controller to control and optimize the testing process. The proposed model uses an adaptive approach to optimize the distributed testing process. This approach is based on Markov Decision Processes (MDP), where a controller is learning an optimal policy based on the MDP model of the specification by using the proposed algorithm. This approach will not only optimise the testing process by eliminating observability and synchronization messages between testers but also it will lead to an adaptive controller that will adapts to the probabilistic behaviour of the distributed testing system by learning to make the optimal decision during test execution.
As prospect, we plan to establish further investigations related to the consistency of our approach. To this end, we conduct actually more empirical studies to implement our algorithms and to evaluate the impact of the proposed approach for revealing errors in real case studies.