Protocol State Fuzzing of DTLS Using Register Automata Learning

Sammanfattning

Protocols are the backbone of all online communication and in order to avoid security vulnerabilities and interoperability issues implementations are required to adhere to the protocol specification. Verifying adherence is however difficult. One method of doing this is state fuzzing, an approach utilizing model learning to produce state machine models of implementations that can be compared to the protocol specification in order to verify conformance. By utilizing register automata (RA) learning this project aims to improve existing state fuzzing approaches with the ability to learn richer models capturing the data flow of the underlying protocol, in this case Datagram Transport Layer Security (DTLS). The goal is to compare RA learning to the existing method of Mealy learning, and answer questions in regards to performance and model detail. To achieve this the ProtocolState-Fuzzer (PSF) tool was updated with RA learning capability courtesy of RALib, thereafter the changes were integrated into the DTLS-fuzzer tool, and finally the DTLS-fuzzer tool was used to learn 3 different DTLS implementations. Experimental evaluation showed that models produced by RA-learning were on average less detailed than those generated by Mealy learning. Furthermore RA-learning performed worse than Mealy-learning for 10 of the 13 experiments ran. A comparison between the number of tests required to learn a model showed that RA-learning uses more tests in 11 of the 13 experiments ran. Thus this project highlights where improvements can be made in RA-learning methodology to facilitate the learning of models with greater detail.