A New Hope for Network Model Generalization
Abstract
Generalizing machine learning (ML) models for network traffic dynamics tends to be considered a lost cause. Hence for every new task, we design new models and train them on model-specific datasets closely mimicking the deployment environments. Yet, an ML architecture called Transformer has enabled previously unimaginable generalization in other domains. Nowadays, one can download a model pre-trained on massive datasets and only fine-tune it for a specific task and context with comparatively little time and data. These fine-tuned models are now state-of-the-art for many benchmarks. We believe this progress could translate to networking and propose a Network Traffic Transformer (NTT), a transformer adapted to learn network dynamics from packet traces. Our initial results are promising: NTT seems able to generalize to new prediction tasks and environments. This study suggests there is still hope for generalization through future research.
Research Area: Network Analysis and Reasoning
People
Talk
BibTex
@INPROCEEDINGS{dietmüller2022network,
isbn = {978-1-4503-9899-2},
copyright = {Creative Commons Attribution 4.0 International},
doi = {10.3929/ethz-b-000577569},
year = {2022-11},
booktitle = {HotNets '22: Proceedings of the 21st ACM Workshop on Hot Topics in Networks},
type = {Conference Paper},
institution = {ETHZ and ETHZ},
author = {Dietmüller, Alexander and Ray, Siddhant and Jacob, Romain and Vanbever, Laurent},
abstract = {Generalizing machine learning (ML) models for network traffic dynamics tends to be considered a lost cause. Hence for every new task, we design new models and train them on model-specific datasets closely mimicking the deployment environments. Yet, an ML architecture called Transformer has enabled previously unimaginable generalization in other domains. Nowadays, one can download a model pre-trained on massive datasets and only fine-tune it for a specific task and context with comparatively little time and data. These fine-tuned models are now state-of-the-art for many benchmarks.We believe this progress could translate to networking and propose a Network Traffic Transformer (NTT), a transformer adapted to learn network dynamics from packet traces. Our initial results are promising: NTT seems able to generalize to new prediction tasks and environments. This study suggests there is still hope for generalization through future research.},
keywords = {Transformer; Packet-level modeling},
language = {en},
address = {New York, NY},
publisher = {Association for Computing Machinery},
title = {A New Hope for Network Model Generalization},
PAGES = {152 - 159},
Note = {21st ACM Workshop on Hot Topics in Networks (HotNets 2022); Conference Location: Austin, TX, USA; Conference Date: November 14-15, 2022; Conference lecture held on November 14, 2022}
}Research Collection: 20.500.11850/577569
Slide Sources: https://gitlab.ethz.ch/projects/41272