A New Hope for Network Model Generalization

HotNets '22: Proceedings of the 21st ACM Workshop on Hot Topics in Networks

Abstract

Generalizing machine learning (ML) models for network traffic dynamics tends to be considered a lost cause. Hence for every new task, we design new models and train them on model-specific datasets closely mimicking the deployment environments. Yet, an ML architecture called Transformer has enabled previously unimaginable generalization in other domains. Nowadays, one can download a model pre-trained on massive datasets and only fine-tune it for a specific task and context with comparatively little time and data. These fine-tuned models are now state-of-the-art for many benchmarks. We believe this progress could translate to networking and propose a Network Traffic Transformer (NTT), a transformer adapted to learn network dynamics from packet traces. Our initial results are promising: NTT seems able to generalize to new prediction tasks and environments. This study suggests there is still hope for generalization through future research.

Research Area: Network Analysis and Reasoning

People

Dr. Alexander Dietmüller
PhD student
2018—2024

Talk

BibTex

@INPROCEEDINGS{dietmüller2022network,
	isbn = {978-1-4503-9899-2},
	copyright = {Creative Commons Attribution 4.0 International},
	doi = {10.3929/ethz-b-000577569},
	year = {2022-11},
	booktitle = {HotNets '22: Proceedings of the 21st ACM Workshop on Hot Topics in Networks},
	type = {Conference Paper},
	institution = {ETHZ and ETHZ},
	author = {Dietmüller, Alexander and Ray, Siddhant and Jacob, Romain and Vanbever, Laurent},
	abstract = {Generalizing machine learning (ML) models for network traffic dynamics tends to be considered a lost cause. Hence for every new task, we design new models and train them on model-specific datasets closely mimicking the deployment environments. Yet, an ML architecture called Transformer has enabled previously unimaginable generalization in other domains. Nowadays, one can download a model pre-trained on massive datasets and only fine-tune it for a specific task and context with comparatively little time and data. These fine-tuned models are now state-of-the-art for many benchmarks.We believe this progress could translate to networking and propose a Network Traffic Transformer (NTT), a transformer adapted to learn network dynamics from packet traces. Our initial results are promising: NTT seems able to generalize to new prediction tasks and environments. This study suggests there is still hope for generalization through future research.},
	keywords = {Transformer; Packet-level modeling},
	language = {en},
	address = {New York, NY},
	publisher = {Association for Computing Machinery},
	title = {A New Hope for Network Model Generalization},
	PAGES = {152 - 159},
	Note = {21st ACM Workshop on Hot Topics in Networks (HotNets 2022); Conference Location: Austin, TX, USA; Conference Date: November 14-15, 2022; Conference lecture held on November 14, 2022}
}

Research Collection: 20.500.11850/577569

Slide Sources: https://gitlab.ethz.ch/projects/41272