An internal schema doc describes an 'Unknown Protocol Identification Database': an Elasticsearch port-asset table tracking active/passive/fused protocol-type guesses and banner text per IP:port, feeding a MySQL clustering pipeline (cluster_info/cluster_task) that groups unclassified traffic by a 'fingerprint' field into named-protocol clusters -- an unsupervised discovery pipeline for identifying and naming new/unknown protocols at scale, distinct from MAAT/AppSketch's signature-matching against already-known protocols.
未知协议识别数据库...active_protocol_type keyword 主动协议类型 unknown...未知协议分析聚类表...fingerprint keyword 指纹信息 protocol keyword 命名协议类型
Defense implications
- A dedicated unsupervised pipeline exists to discover and name previously-unknown protocols by clustering fingerprint+banner+active/passive probe data at IP:port scale -- a brand-new or rare pluggable transport does not stay invisible merely by lacking an existing signature; sufficiently prevalent unknown traffic gets clustered and becomes a candidate for a hand-written signature regardless of whether it has been publicly documented yet.
Related findings
Reveals internal architecture of TSG's traffic classification engine: the 'sapp' process (binary at /opt/tsg/sapp/sapp, version sapp-4.2.90) uses a packet-I/O layer called 'marsio', a custom app-identification plugin 'app_sketch_local' (identify_app_by_tcp_payload / identify_app_by_user_define_attributes), and an embedded LuaJIT ('libelua') for user-defined protocol signatures (e.g. custom Modbus detection) — concurrent LUA script loading crashes the classification worker.
MESA Lab engineering logs (2020) describe development and TSG-environment testing of a SAPP plugin that identifies encrypted audio/video traffic and distinguishes on-demand from live streams. The workflow uses Fiddler (a TLS MITM proxy) to decrypt traffic client-side and auto-label training data for the underlying ML model, and required a dlopen() workaround to load libpython.so for the model at runtime.
An internal TSG operations/troubleshooting manual lays out TSG's full traffic pipeline (NIC -> mrzcpd/marsio capture driver -> sapp DPI engine -> firewall/proxy(KNI->TFE)/active-defense/WAN-NAT policy branches) and shows engineers using maat_redis_tool to pull the live Redis-synced blocking policy tables (TSG_SECURITY_COMPILE, TSG_OBJ_IP_ADDR, TSG_OBJ_APP_ID) and filter them by numeric policy/object ID to debug why a block rule isn't firing.
The internal 'MAAT网络流处理配置统一描述框架' engineering manual (v3.1.20, author 郑超, 2021) documents MAAT's Redis-synced rule-compilation framework and its RuleScan/Hyperscan-based pattern-matching engine (libmaatframe.so / librulescan), and records that RuleScan's fast-scan feature caused a production outage on 2019-03-19 and has been disabled ever since.
An internal design note for a 'Shadowsocks traffic parsing/restoration module' describes decrypting captured Shadowsocks payloads back to the original HTTP request/response, given a known pre-shared key (AES-256-CFB, MD5-derived key, IV embedded in the stream).
The same internal research note's second research point develops an ML-based detector for Geneva-style automated censorship-evasion traffic; simple flow-level features (flow size, max packet size, RST/SYN/FIN flag counts, forward init-window bytes, inter-arrival timing) achieve near-perfect (ROC-AUC ~1.00) classification of Geneva-generated evasion traffic against CICIDS2017 and MAWI backbone background traffic using decision trees, LightGBM, XGBoost and random forest, with abnormal flow size (~150 bytes vs. 1000-30000 bytes typical) identified as the single most discriminative feature.