Traffic-shape / statistical fingerprinting
The same internal research note's second research point develops an ML-based detector for Geneva-style automated censorship-evasion traffic; simple flow-level features (flow size, max packet size, RST/SYN/FIN flag counts, forward init-window bytes, inter-arrival timing) achieve near-perfect (ROC-AUC ~1.00) classification of Geneva-generated evasion traffic against CICIDS2017 and MAWI backbone background traffic using decision trees, LightGBM, XGBoost and random forest, with abnormal flow size (~150 bytes vs. 1000-30000 bytes typical) identified as the single most discriminative feature.
A dedicated internal "运营商前端分析团队" (Carrier Front-end Analysis Team) produces weekly, carrier-specific traffic-analysis reports explicitly titled "新疆移动流量分析报告" (Xinjiang Mobile Traffic Analysis Report) and "新疆联通流量分析报告" (Xinjiang Unicom Traffic Analysis Report), with instances dated from at least July 2022 through March 2024. One instance (Xinjiang Mobile, 2023-05-15) reports total carrier bandwidth of 4.4Tbps (1Tbps access bandwidth), 27.34PB of weekly throughput, 123 distinct identified application-layer protocols, and a section (4.1, pages 17+) explicitly tracking server IP/location for 28 named overseas apps (BBC, Discord, ESPN, Facebook, Gmail, Google, Hulu, Instagram, Netflix, Reddit, Skype, Telegram, Twitter, Uber, WhatsApp, Wikipedia, YouTube, Zoom, etc.).
A 2020 MESA Lab monthly report describes building a sapp plugin that extracts packet-sequence features specifically "for DoH (DNS-over-HTTPS) service discovery," alongside a broader CSTNET DoH measurement-report effort and configuring a DNS-to-DoH gateway -- confirming sapp is used to fingerprint DoH traffic via statistical sequence features rather than plaintext DNS content.
Internal MESA Lab reading notes dissect the USENIX 2024 paper on fingerprinting obfuscated proxies via encapsulated TLS handshakes, highlighting its protocol-agnostic packet-size-3-gram-plus-Mahalanobis-distance-over-bursts classifier, which the paper's own mid-size-ISP deployment reliably fingerprinted across shadowsocks, vmess, trojan, and vless-family configurations at false-positive rates the notes explicitly say the annotator estimates the GFW would find operationally acceptable (<0.6%). The notes flag the technique's main gaps as: no public source code, sharply reduced true-positive rate under connection multiplexing (10-30% vs. 60-80% unmultiplexed), and no evaluation against UDP/QUIC.
A 2023 master's thesis from the Institute of Information Engineering, Chinese Academy of Sciences (MESA Lab's parent institute) develops an online encrypted-video-stream title identification method using response-time distribution and chunk-fingerprint matching that survives CDN node switching, reporting 96.19% title-identification accuracy using only 6 observed video chunks, without decrypting traffic.
A thesis-project assignment for hidden-service (VPN/Proxy/Tor) identification via heterogeneous graph neural networks on flow logs sources its one day of training data from a database explicitly named 'tsg_galaxy_p19' — tying the internal TSG naming convention to the taxonomy's P19/WMS-UTR Pakistan site codename — accessed over an internal 'Information Harbor' (信息港) VPN, with ground-truth hidden-service IP labels supplied by the commercial IP-intelligence service spur.us.
A master's thesis-in-progress supervised within this ecosystem (advisor 杨嵘, senior engineer) is explicitly tied to the national key R&D program "海量公害 网页、图片、视频流量识别技术" (massive nuisance webpage/image/video traffic identification technology) and a VoIP engineering project; it builds cross- platform (YouTube/Facebook/Bilibili) encrypted-video identification that matches a single per-video traffic fingerprint across different CDNs/resolutions/codecs via trend-similarity, fuzzy-search, and meta-learning matching, explicitly framed around detecting "非法视频" (illegal/harmful video) spread across platforms.
A 2024 MESA thesis proposal targets detecting "cross-border apps that illegally transmit personal information" inside encrypted traffic without decryption, by discovering app-specific tracking identifiers as <domain,parameter,value,interval> tuples and clustering flow structural similarity (Euclidean distance in a feature space) to separate device identifiers from background noise via a two-layer recognition model.
A weekly "新疆移动流量分析报告" (Xinjiang Mobile Traffic Analysis Report), authored by the "运营商前端分析团队" (Carrier Front-End Analysis Team), directly ties the SNI/Server-IP overseas- APP report format to the China Mobile Xinjiang branch specifically, and states the pipeline identifies 126 distinct application-layer protocols including multi-layer tunnel nesting such as STUN.DTLS and STUN.RTP.RTCP.DTLS — i.e. it decomposes and classifies nested WebRTC-style transport stacks, not just top-level TLS/QUIC.
Acceptance-test cases for an unnamed internal system show it curates 10,000+ DNS rules tracked for 30+ days each, runs hourly business-anomaly detection and bot-vs-human traffic classification per flagged domain (target accuracy/recall ≥80-95%), and maintains a resulting "abnormal IP" blocklist database of 2,000+ entries — an automated domain/IP flagging pipeline structurally similar to (but not confirmed identical to) the taxonomy's CyberNarrator blocklist-building pattern.
A MESA Lab research project (wujiating/fingerprinting) builds a DoH (DNS-over-HTTPS) traffic fingerprinting pipeline: captured pcaps of DoH sessions from Chrome/Firefox/Edge on Windows10/Ubuntu, packet-timing/size feature extraction via tsfresh, and an n-gram classifier — ML-based traffic analysis to identify DoH usage and likely client browser/OS from traffic shape alone.
Multiple domestic IDC traffic-monitoring reports rank "Freegate" (a well-known GFW-circumvention tool) as a distinct, named top-10 application by traffic volume alongside Bytedance/Tencent/Alibaba/Baidu, with peak 18.92 Gbps and average 5.9 Gbps in one 2022-07-07 report -- confirming Freegate has its own dedicated app-ID classifier tracked at carrier-comparable traffic scale, not merely a low-volume/rare signature.
An internal "网站/应用资源测绘" (website/app resource-mapping) presentation describes active EDNS-Client-Subnet DNS probing from many simulated geographic vantage points to map Google's and Facebook's global service-IP distribution by country/province, a mobile-app pipeline combining UI automation with MITM-proxy/ SSL-pinning-bypass to extract button-to-URL mappings from decrypted app traffic, and a "网站指纹审查" section that explicitly documents domain fronting as an effective evasion of its own DNS/SNI/Host-based website-fingerprint detection, alongside literature-based behavioral/ML website-fingerprinting intended to survive the QUIC/ECH/DoH transition.
An internal research writeup ("研究点二：基于GRU神经网络的共享接入IP检测技术") builds a GRU/CNN model over sequences of TLS JA3/SNI/session-ticket and HTTP cookie fingerprints, trained on 6 days / 155GB of mirrored traffic captured from an internal gateway named "华严网关" (Huayan Gateway), to determine whether a single source IP represents one device or several devices sharing that IP (e.g. behind a NAT/proxy), reporting precision 0.844 / recall 0.874 / F1 0.859.
Recurring "entry/exit overall traffic monitoring" (出入口整体流量监测报告) and per-provider "IDC quality monitoring" reports show China domestic gateway links running at up to 1.68 Tbps aggregate (24 links) with per-app (Bytedance, Tencent, Kuaishou, Alibaba, Baidu, Apple, Netease, Pinduoduo, Xiaomi, bilibili) traffic-share, TCP-handshake-latency, and packet-loss-rate breakdowns generated on a routine (near-daily/weekly) automated cadence — dozens of these reports recur across the corpus at different dates.
Two IIE graduate-research repos (cuiyiming/gradproj, a 2019-2020 master's thesis project citing NDSS'17 TLS-interception-measurement and TLS-client-identification papers; daxiaoxu/xmr_bsexpr2, a 2022 project with GRU-based sequence classifiers over TCP/DNS flow JSON and deleted docs on TLS1.3 and Tencent's proprietary MMTLS protocol) document the ML feature-engineering methodology (TLS certificate length, handshake message sequences, JA3-style statistics, Markov-chain packet-size/timing models, GRU sequence models) that plausibly underlies production classifiers (e.g. stellar's later JA4/JA4S support, MESA_sts's randomness checks).
A MESA Lab student research report catalogs the GFW's known Shadowsocks-detection methodology (passive detection via first-packet length/entropy; active probing triggered after as few as 13 legitimate client connections, typically within seconds of the first legitimate connection) alongside six published ML-based Shadowsocks traffic-classification techniques (a packet-size-image CNN at >98% accuracy, random-forest on flow/host/DNS-behavior features, PCA-Pearson feature selection), compiled as apparent background research for in-house detection work.
Grityu/model_duplication ("five modes duplication", 2023) built and evaluated ML traffic classifiers (Knn_test.py, ScenarioA.pkl, features.json) using CICFlowMeter flow-feature extraction over labeled pcap captures of major commercial VPN/circumvention tools — Psiphon (including a specific "psiphon_operation" capture), Surfshark over OpenVPN TCP/UDP, TorGuard over OpenVPN UDP and "OpenVPN over SSL", and Hotspot Shield VPN — directly evidencing GFW-side R&D on flow-level statistical fingerprinting of specific VPN products.
A MESA research-log entry details a NAT/shared-connection identification methodology combining TCP/IP fingerprinting (IP-ID, TTL, DF, window size, MSS, TCP-option ordering -- p0f-style), HTTP User-Agent diversity, TLS/SSL JA3 fingerprint diversity, and traffic statistical features (concurrent-TCP-connection count, idle-time jitter, upstream/downstream ratio stability, DNS query frequency) per endpoint over rolling time windows, with detection methods spanning direct UA inspection, threshold statistics, ML classifiers (random forest/SVM), and per-window entropy jumps across the fingerprint features.
A MESA Lab git repo (wangmeiqi/obfs4_meek_snowflake) trains closed-world Deep Fingerprinting (DF) website-fingerprinting classifiers (ClosedWorld_DF_NoDef.py) using pre-trained Keras models specifically for Tor's obfs4 pluggable transport, meek domain-fronting, and Snowflake -- the same transport families Lantern's own circumvention stack draws on.
A performance bug at a domestic Xinjiang test site (25-70Gbps) traced packet-processing lock contention to a plugin named "tsg_vulpes" calling an ONNX Runtime model for real-time "encrypted voice recognition" on live traffic; disabling this ML classifier resolved packet loss, indicating it's an optional, performance-costly add-on.
During QuarkVPN feature extraction, Geedge found client/server roles over UDP distinguishable by port-number heuristic (server's fixed ports 59001/59002/60000 exceed the client's random port), though this heuristic is currently causing misidentification, filed as an open bug.
Geedge runs a video-content-fingerprinting pipeline: a crawler pulls YouTube videos by topic, routed through mitmproxy with a custom trusted root CA to intercept TLS and capture per-video traffic, targeting ~100 captures per topic.
A patent draft (applicant not present in the extracted text, but topically and methodologically identical to MESA Lab's SAPP audio/video-identification plugin work in the same corpus) describes a content-level encrypted-video identification system: an SNI-triggered flow classifier groups downlink ACK-aligned packets into 'chunks', converts the chunk-size sequence into a long/short word-frequency signature, and matches it in O(1) against a fingerprint database built by actively crawling target sites through a MITM proxy. Tested on 1,000 YouTube videos captured from 5 locations, it reaches 96.19% accuracy needing only 6 online chunks at 3.33us per match, 90x+ faster than three prior published methods it benchmarks against.
A standalone DPI plugin (liuchang/pkt_seq_matcher, built against Hyperscan and a Ragel-generated state machine) computes per-flow packet-payload-length sequences and matches them against compiled patterns, explicitly excluding zero-payload packets from the length calculation -- a packet-length statistical fingerprinting approach, the kind of technique used to identify fully-encrypted/obfuscated pluggable-transport traffic by its handshake length signature rather than by content.
A MESA Lab repo (daxiaoxu/xmr_bsexpr3) contains standalone detection/fingerprinting scripts specifically targeting QQ (Mobile and Windows clients) and WeChat, alongside a script named PSK_LifeTime.py, suggesting TLS session-resumption / PSK-ticket lifetime is being explored as a traffic-fingerprinting feature for identifying or tracking these apps' encrypted sessions.
MESA Lab meeting notes (Nov-Dec 2021) document development and a staged production rollout of a new sapp plugin that identifies encrypted video streams via "burst" (packet-timing/size-burst) traffic features designed to generalize across varying network conditions, trained with machine learning, with a first burst-rule version scheduled for live deployment and sapp modified to log additional features for continued training.
A 2021 MESA meeting note on an "encrypted video stream content identification" experiment records collecting Tencent Video traffic through SAPP, a SAPP bug where retransmitted packets are not counted during packet processing, and next steps to study YouTube's video transmission patterns and circumvention/router-VPN techniques -- explicitly framed as choosing collection targets "from a regulatory/censorship standpoint, favoring sites with weaker self-censorship."
An internal MESA Lab research survey dedicated to Snowflake reviews five external papers on fingerprinting/blocking Snowflake and WebRTC-based transports, including a 2023 CSCWD paper that fingerprints Snowflake's broker-contact requests (disguised as HTTPS) via packet size, direction, timing, and network speed to distinguish them from ordinary web requests, and a 2020 paper claiming 100% identification of Snowflake DTLS handshakes against Facebook Messenger/Hangouts/ Discord WebRTC traffic in a closed-world test — annotated throughout with the MESA reviewer's own critical commentary on each method's weaknesses (e.g. sensitivity to user geography/network, only tested at host/LAN vantage points rather than backbone).
A ClickHouse SQL query against table tsg_galaxy_v3.session_record_local implements a heuristic for flagging likely proxy/circumvention-tool usage: within 5-second windows, it flags any client IP that connects to 30+ distinct server IPs on high ports (>=10000) with small, tightly-bounded packet counts (6-15 sent, 4-15 received) where the session's application-layer protocol is unclassified (app_transition = 'unknown'). "tsg_galaxy_v3" indicates a named TSG data-warehouse/analytics layer distinct from the sapp/maat real-time inspection path.
A CAS-institute thesis ("面向加密场景的视频入口发现关键技术研究") builds a system that associates known encrypted video-resource traffic back to its hosting webpage/app "entry" domain even when the video itself is served from generic CDN infrastructure, using co-occurrence/sequence mining plus active-learning ranking (XGBoost) partly trained on plaintext-flow ground truth; the author's prior project record lists a "某音视频节目库项目" funded under the national "242" information-security special program (2019–2020), and the thesis is explicitly framed as enabling detection of "有害视频" (harmful video) sources.
A CAS-institute master's thesis ("基于IP相似性分析的VPN服务识别技术研究") builds a VPN-server-identification system combining active TCP/UDP port probing (SYN scan on ports 1194 OpenVPN, 500/4500 IPSec, 1701 L2TP, 22, 443, 51820 WireGuard, 992, 4090, 655; UDP scan on 80/53/25/465/110/143/389/21/3389/445/69/3306/6379) with passive-traffic "IP similarity" graph analysis, reaching 92.44% online-test accuracy and supporting dynamic feature-library updates as services change.
wujiating/detection (by the same MESA Lab researcher behind wujiating/censorship_detection, a translated censorship-detection literature survey) is a CICFlowMeter-based ML traffic classifier trained on the public ISCX VPN-nonVPN dataset plus custom-captured DoH and generic web pcaps, organized into explicit closed-world (CW) and open-world (OW) evaluation splits — evidence of dedicated open-world DoH-traffic classification research at MESA Lab.
Recurring weekly '新疆移动流量分析报告' / '新疆联通流量分析报告' (Xinjiang Mobile / Xinjiang Unicom traffic analysis reports) document a domestic monitoring deployment covering China Mobile Xinjiang (total capacity 4.4Tbps, ~1Tbps actively monitored, observed peak 1275.49Gbps) and China Unicom Xinjiang, breaking traffic into intra-Xinjiang, domestic, and overseas flows and reproducing the same per-foreign-platform (YouTube/Google/Facebook/Twitter/Instagram/Telegram/WhatsApp/etc.) server-IP breakdown used in the national TSG SNI reports, at a carrier-and-province-specific granularity.
MESA Lab researchers built and iterated a Scapy-based website-fingerprinting classifier that identifies specific YouTube videos from encrypted traffic by matching the packet-size sequence of thumbnail-image requests (triggered on SNI yt3.googleusercontent.com / yt3.ggpht.com) against a pre-built reference dataset (dataset_24_youtube_fingerprints.csv), scoring candidate matches on sequence overlap and total-payload-size ratio against fixed thresholds (0.65 / 0.85).
Thesis-defense revision notes for a MESA-adjacent CAS-institute master's thesis describe a YouTube video-title identification system that decrypts HTTPS video traffic between a crawler and video servers via mitmproxy, then fingerprints each video from per-chunk HTTP response timing/size derived from a corpus of 38,000 YouTube video chunks (identification method: response-time-distribution based).