ML / statistical classifier

The same internal research note's second research point develops an ML-based detector for Geneva-style automated censorship-evasion traffic; simple flow-level features (flow size, max packet size, RST/SYN/FIN flag counts, forward init-window bytes, inter-arrival timing) achieve near-perfect (ROC-AUC ~1.00) classification of Geneva-generated evasion traffic against CICIDS2017 and MAWI backbone background traffic using decision trees, LightGBM, XGBoost and random forest, with abnormal flow size (~150 bytes vs. 1000-30000 bytes typical) identified as the single most discriminative feature.

A 2020 MESA Lab monthly report describes building a sapp plugin that extracts packet-sequence features specifically "for DoH (DNS-over-HTTPS) service discovery," alongside a broader CSTNET DoH measurement-report effort and configuring a DNS-to-DoH gateway -- confirming sapp is used to fingerprint DoH traffic via statistical sequence features rather than plaintext DNS content.

MESA Lab researcher notes on an encrypted-video-identification project describe adding SSL-layer information output to more precisely trace a flow's true source/identity, and considering reinforcement learning so the identification model adapts as network conditions change, working within/around sapp's plugin limitations.

An internal schema doc describes an 'Unknown Protocol Identification Database': an Elasticsearch port-asset table tracking active/passive/fused protocol-type guesses and banner text per IP:port, feeding a MySQL clustering pipeline (cluster_info/cluster_task) that groups unclassified traffic by a 'fingerprint' field into named-protocol clusters -- an unsupervised discovery pipeline for identifying and naming new/unknown protocols at scale, distinct from MAAT/AppSketch's signature-matching against already-known protocols.

A thesis-project assignment for hidden-service (VPN/Proxy/Tor) identification via heterogeneous graph neural networks on flow logs sources its one day of training data from a database explicitly named 'tsg_galaxy_p19' — tying the internal TSG naming convention to the taxonomy's P19/WMS-UTR Pakistan site codename — accessed over an internal 'Information Harbor' (信息港) VPN, with ground-truth hidden-service IP labels supplied by the commercial IP-intelligence service spur.us.

A master's thesis-in-progress supervised within this ecosystem (advisor 杨嵘, senior engineer) is explicitly tied to the national key R&D program "海量公害 网页、图片、视频流量识别技术" (massive nuisance webpage/image/video traffic identification technology) and a VoIP engineering project; it builds cross- platform (YouTube/Facebook/Bilibili) encrypted-video identification that matches a single per-video traffic fingerprint across different CDNs/resolutions/codecs via trend-similarity, fuzzy-search, and meta-learning matching, explicitly framed around detecting "非法视频" (illegal/harmful video) spread across platforms.

A 2024 MESA thesis proposal targets detecting "cross-border apps that illegally transmit personal information" inside encrypted traffic without decryption, by discovering app-specific tracking identifiers as <domain,parameter,value,interval> tuples and clustering flow structural similarity (Euclidean distance in a feature space) to separate device identifiers from background noise via a two-layer recognition model.

Acceptance-test cases for an unnamed internal system show it curates 10,000+ DNS rules tracked for 30+ days each, runs hourly business-anomaly detection and bot-vs-human traffic classification per flagged domain (target accuracy/recall ≥80-95%), and maintains a resulting "abnormal IP" blocklist database of 2,000+ entries — an automated domain/IP flagging pipeline structurally similar to (but not confirmed identical to) the taxonomy's CyberNarrator blocklist-building pattern.

A MESA Lab research project (wujiating/fingerprinting) builds a DoH (DNS-over-HTTPS) traffic fingerprinting pipeline: captured pcaps of DoH sessions from Chrome/Firefox/Edge on Windows10/Ubuntu, packet-timing/size feature extraction via tsfresh, and an n-gram classifier — ML-based traffic analysis to identify DoH usage and likely client browser/OS from traffic shape alone.

A 2024 CAS/IIE master's mid-term thesis report, "面向DPI中间件的探测行为识别关键技术研究" (Key Technologies for Identifying Probing Behavior Targeting DPI Middleboxes), builds and evaluates a system that detects and fingerprints active-measurement traffic from OONI, Censored Planet, and GFWatch/GFWeb (via their server-contact patterns, distinct-domain-count thresholds, and response-timing signatures — e.g. flagging a probing srcIP once it queries ≥500-2000 distinct domains at one dstIP), and separately trains a graph-neural-network classifier on raw packet bytes to detect Geneva/SymTCP-style automated censorship-evasion probes, explicitly framed as reducing the DPI middlebox's exposure of its own characteristics to circumvention researchers.

An internal research writeup ("研究点二：基于GRU神经网络的共享接入IP检测技术") builds a GRU/CNN model over sequences of TLS JA3/SNI/session-ticket and HTTP cookie fingerprints, trained on 6 days / 155GB of mirrored traffic captured from an internal gateway named "华严网关" (Huayan Gateway), to determine whether a single source IP represents one device or several devices sharing that IP (e.g. behind a NAT/proxy), reporting precision 0.844 / recall 0.874 / F1 0.859.

A CAS-institute thesis on differentiating human vs. automated ("machine") network behavior deploys an HTTP/SSL key-field extraction and app-identification system at a live network gateway (mirrored traffic) built on a modified nDPI (extended to recognize 108 application-layer protocols) plus a custom rule-scan module whose region/group/compile config hierarchy and terminology closely parallel MAAT's separately-documented architecture, suggesting shared lineage between this academic prototype and MAAT's production rule engine.

Two IIE graduate-research repos (cuiyiming/gradproj, a 2019-2020 master's thesis project citing NDSS'17 TLS-interception-measurement and TLS-client-identification papers; daxiaoxu/xmr_bsexpr2, a 2022 project with GRU-based sequence classifiers over TCP/DNS flow JSON and deleted docs on TLS1.3 and Tencent's proprietary MMTLS protocol) document the ML feature-engineering methodology (TLS certificate length, handshake message sequences, JA3-style statistics, Markov-chain packet-size/timing models, GRU sequence models) that plausibly underlies production classifiers (e.g. stellar's later JA4/JA4S support, MESA_sts's randomness checks).

The leak includes saved copies of external research directly relevant to circumvention detection: a Chinese-Academy-of-Sciences paper (FS-Net) proposing an end-to-end recurrent-neural-network model for encrypted traffic classification (99.14% TPR / 0.05% FPR across 18 applications), and a 2008 Shanghai Jiao Tong University paper reverse-engineering UltraSurf's client via dynamic disassembly to recover its proxy protocol, encryption scheme, and network topology -- indicating MESA Lab/Geedge researchers maintain a working reference library spanning both ML-based traffic classification methodology and reverse-engineering methodology for circumvention client software, rather than relying solely on in-house techniques.

An internal system-design document for a "Web Fingerprint" module specifies a mirrored-traffic system that identifies specific web pages a monitored individual visits over encrypted connections (the worked example given is a specific politically-related YouTube channel homepage) and specific search-engine keywords typed into Google search (the worked example target keyword given is "FLG", i.e. Falun Gong), with accuracy targets of >=90% page-identification precision and >=95% keyword recall, feeding a downstream "reputation" scoring module. A companion Python implementation with a real "dataset_24_youtube_ fingerprints.csv" dataset performs the YouTube-page fingerprinting using picture-count and request-size-sequence features, and is explicitly scoped in its file path to a "特定开放通道" (a specific open/circumvention channel).

A MESA Lab student research report catalogs the GFW's known Shadowsocks-detection methodology (passive detection via first-packet length/entropy; active probing triggered after as few as 13 legitimate client connections, typically within seconds of the first legitimate connection) alongside six published ML-based Shadowsocks traffic-classification techniques (a packet-size-image CNN at >98% accuracy, random-forest on flow/host/DNS-behavior features, PCA-Pearson feature selection), compiled as apparent background research for in-house detection work.

Grityu/model_duplication ("five modes duplication", 2023) built and evaluated ML traffic classifiers (Knn_test.py, ScenarioA.pkl, features.json) using CICFlowMeter flow-feature extraction over labeled pcap captures of major commercial VPN/circumvention tools — Psiphon (including a specific "psiphon_operation" capture), Surfshark over OpenVPN TCP/UDP, TorGuard over OpenVPN UDP and "OpenVPN over SSL", and Hotspot Shield VPN — directly evidencing GFW-side R&D on flow-level statistical fingerprinting of specific VPN products.

A MESA research-log entry details a NAT/shared-connection identification methodology combining TCP/IP fingerprinting (IP-ID, TTL, DF, window size, MSS, TCP-option ordering -- p0f-style), HTTP User-Agent diversity, TLS/SSL JA3 fingerprint diversity, and traffic statistical features (concurrent-TCP-connection count, idle-time jitter, upstream/downstream ratio stability, DNS query frequency) per endpoint over rolling time windows, with detection methods spanning direct UA inspection, threshold statistics, ML classifiers (random forest/SVM), and per-window entropy jumps across the fingerprint features.

A MESA Lab git repo (wangmeiqi/obfs4_meek_snowflake) trains closed-world Deep Fingerprinting (DF) website-fingerprinting classifiers (ClosedWorld_DF_NoDef.py) using pre-trained Keras models specifically for Tor's obfs4 pluggable transport, meek domain-fronting, and Snowflake -- the same transport families Lantern's own circumvention stack draws on.

TSG's Psiphon3 blocking (Ethiopia/E21 site) uses a dynamically-learned "Top SNI" / "Top Server IP" allowlist meant to avoid collaterally blocking shared infrastructure Psiphon3 also rides on (e.g. Google); a bug in the learning pipeline (SNI values under 3 bytes rolled back the whole DB write transaction) let the allowlist silently go stale, causing Google traffic to be misidentified and blocked as Psiphon3.

A performance bug at a domestic Xinjiang test site (25-70Gbps) traced packet-processing lock contention to a plugin named "tsg_vulpes" calling an ONNX Runtime model for real-time "encrypted voice recognition" on live traffic; disabling this ML classifier resolved packet loss, indicating it's an optional, performance-costly add-on.

For the M22 project (domestic-China-labeled training ticket, but M22 is elsewhere confirmed as a Myanmar deployment), Geedge explicitly instructs trainers to describe 'Cyber Narrator' only as a black-box ML capability, to avoid revealing the internal domain tsg.bj.internal.geedge.net, and to omit the company logo from training materials — i.e. deliberate concealment of Geedge's involvement and internal infrastructure from the training audience.

TSG engineers explicitly acknowledged to an Ethiopia customer that the system's application-identification statistics over-count Psiphon3 and Freegate due to misidentification, inflating their apparent traffic share (e.g. Psiphon3 appearing in the application Top-2 despite the customer reporting no active blocking of it), and that Netflix's ranking also varies drastically by sort metric (bytes vs. sessions vs. unique client IP).

TSG runs an automated program named 'vpn-thwarting' that calls the CM management API hourly to push freshly-learned Psiphon3 server IPs into a live blocklist object (dynamic_psiphon_ip); at the time of this incident the object held roughly 70,000-73,000 IPs, sourced from a companion automated-learning system referred to as 'CN'.

Reveals internal architecture of TSG's traffic classification engine: the 'sapp' process (binary at /opt/tsg/sapp/sapp, version sapp-4.2.90) uses a packet-I/O layer called 'marsio', a custom app-identification plugin 'app_sketch_local' (identify_app_by_tcp_payload / identify_app_by_user_define_attributes), and an embedded LuaJIT ('libelua') for user-defined protocol signatures (e.g. custom Modbus detection) — concurrent LUA script loading crashes the classification worker.

MESA Lab engineering logs (2020) describe development and TSG-environment testing of a SAPP plugin that identifies encrypted audio/video traffic and distinguishes on-demand from live streams. The workflow uses Fiddler (a TLS MITM proxy) to decrypt traffic client-side and auto-label training data for the underlying ML model, and required a dlopen() workaround to load libpython.so for the model at runtime.

MESA Lab meeting notes (Nov-Dec 2021) document development and a staged production rollout of a new sapp plugin that identifies encrypted video streams via "burst" (packet-timing/size-burst) traffic features designed to generalize across varying network conditions, trained with machine learning, with a first burst-rule version scheduled for live deployment and sapp modified to log additional features for continued training.

A UCAS master's thesis repo by an Institute of Information Engineering (Chinese Academy of Sciences)-affiliated researcher (chenguanlin_thesis, referencing the PACK algorithm, content-based classification, and a YouTube test image) and a related MESA Lab repo by the same author (chenguanlin/td_evaluation) that builds 'video ID' fingerprints from traffic (vedio_id_build.c) and evaluates detection accuracy with formal mistake/lost-rate statistics together indicate applied R&D on identifying specific video content from traffic patterns -- a content-fingerprinting capability beyond simple protocol classification.

A CAS-institute thesis ("面向加密场景的视频入口发现关键技术研究") builds a system that associates known encrypted video-resource traffic back to its hosting webpage/app "entry" domain even when the video itself is served from generic CDN infrastructure, using co-occurrence/sequence mining plus active-learning ranking (XGBoost) partly trained on plaintext-flow ground truth; the author's prior project record lists a "某音视频节目库项目" funded under the national "242" information-security special program (2019–2020), and the thesis is explicitly framed as enabling detection of "有害视频" (harmful video) sources.

hezhengjie/videoportaldetection is a Spark/ClickHouse-backed MESA Lab pipeline (cn.ac.iie package) that mines DNS/HTTP/SSL/AV traffic logs with frequent-pattern-mining and a precision/recall evaluation module to automatically discover new audio/video-service domains (source resource file named "AV domains to be detected"), i.e. an automated FQDN-discovery system for media-streaming services that likely feeds a blocklist/classification pipeline rather than relying on static signature lists alone.

A CAS-institute master's thesis ("基于IP相似性分析的VPN服务识别技术研究") builds a VPN-server-identification system combining active TCP/UDP port probing (SYN scan on ports 1194 OpenVPN, 500/4500 IPSec, 1701 L2TP, 22, 443, 51820 WireGuard, 992, 4090, 655; UDP scan on 80/53/25/465/110/143/389/21/3389/445/69/3306/6379) with passive-traffic "IP similarity" graph analysis, reaching 92.44% online-test accuracy and supporting dynamic feature-library updates as services change.

Two versions of an academic active-probing VPN-server-detection paper ("VPNChecker"/"VPNSniffer", WWW'24) held in this corpus specifically profile Psiphon3: Psiphon3 servers account for the single largest share (6.64%) of labeled VPN servers in the authors' ISP dataset, most respond to an OpenVPN probe by silently timing out rather than returning the standard OpenVPN response (a "probe-resistant" behavior the paper explicitly attributes to Psiphon3), and Psiphon3 servers share a small number of characteristic "Probing Port Combinations" (e.g. {443,53,22}, {443,554,22}) across the vendor's fleet that the paper's graph-based classifier uses to link servers together.

An internal talk deconstructing MESA Lab's own WWW2024 paper "Identifying VPN Servers through Graph-Represented Behaviors" (VPNTracker) confirms it was built on the group's own internal production log data ("组内数据"), and details its core active-probing feature: "Stealth Ports" (TCP ports observed open in passive traffic that refuse/ignore active probes) combined with vendor-specific patterns in the error responses VPN servers return to out-of-protocol active probes (e.g. a DNS query sent to the port), fed into a graph neural network over client-IP/server-IP/port relationships; framed explicitly as anti-geo-fraud tooling against services like ChatGPT, Netflix, and NetEase Cloud Music.

wujiating/detection (by the same MESA Lab researcher behind wujiating/censorship_detection, a translated censorship-detection literature survey) is a CICFlowMeter-based ML traffic classifier trained on the public ISCX VPN-nonVPN dataset plus custom-captured DoH and generic web pcaps, organized into explicit closed-world (CW) and open-world (OW) evaluation splits — evidence of dedicated open-world DoH-traffic classification research at MESA Lab.

TSG produces recurring monthly per-application traffic breakdown reports for a Xinjiang deployment, splitting generic protocol buckets (bmff/http2/mpegts) down to specific apps (Kuaishou, Douyin, JD, Vivo services), delivered to the customer.

The daxiaoxu/xmr_bsexpr1 repo is a 3,500+ file dataset of raw PCAP captures and per-device "SUB_FEATUREs" text extracts spanning a wide range of real consumer devices (Honor Magic2, iPad Air2/mini2, LG G7 ThinQ, MacBook Air 2021, MacOS 15, RedMi Note9, Windows 7/10/11), consistent with a device/OS traffic-fingerprinting feature-extraction pipeline feeding TSG's app/device identification signatures.

A 2024 student-thesis project inside MESA Lab (chengyifei/yy_strategy_adjust, presentation titled "毕设交流_大模型策略控制" / "Thesis exchange: large-model strategy control") wires Moonshot AI's Kimi LLM into a "strategy adjustment" module via kimi_main.py/kimi_strategy.py/kimi_strategy_client.py and a content_role_system.txt role/prompt file, indicating exploratory R&D on using an LLM to drive dynamic policy/strategy decisions. Framed as a graduation-project exercise, not confirmed production use.