Analyzing the Signature Algorithm in a Certain App's Android 9.43.1 Client

This article analyzes request signing in Xiaohongshu Android 9.43.1 (build 9431801), including the device identifier, MUA, SIG, S1, Shield, and the key exchange required by Shield. All constants and binary layouts described here are specific to this version.

MUA, SIG, and S1 are handled by the libtiny processing chain, while Shield and main_hmac decryption correspond to libxyass. The discussion below proceeds through input construction, cryptographic transformations, and output formats. Here, native refers to the native libraries in this version; emulator comparisons represent only the library output for the given environment inputs.

Below, || denote byte concatenation, XOR denote bitwise XOR; u32le, u32be respectively denote little-endian and big-endian encodings of 32-bit integers. Truncation in integer operations is noted separately.

This article is a reverse-engineering record of the client's signature implementation, limited to the local algorithms and message formats in Android 9.43.1 (build 9431801). The formulas and constants describe only how this version constructs request headers; they do not provide a method for unauthorized access, bypassing risk controls, or bulk scraping. Server-side policies, device environments, and later versions may all invalidate these conclusions; collection branches and real-device states not covered here are not inferred from a single successful response.

0. Overall signature flow

A content request involves the following fields:

SIG and S1 use the same canonical string, defined below as canonical:

METHOD\nPATH\nQUERY\nhex(SHA256(body))\nMUA

Here, METHOD is uppercase; PATH excludes the domain; QUERY is the original string after the question mark, without further decoding or sorting and without the question mark itself; body is the bytes actually sent; SHA-256 uses 64 lowercase hexadecimal characters; and MUA retains its complete format, including the trailing period. The five items are separated by one LF byte (0x0A), with no extra line break at the end. In the expression above, \n denotes a line break, not a backslash followed by the letter n; when query is empty, the corresponding blank line is retained.

For a GET request, body is empty, and its SHA-256 is:

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Shield uses a separate input: path, query, the specified request headers, and body are concatenated directly in a fixed order. The SIG, S1, and MUA headers are not included directly in the HMAC input.

A verified cold-start client flow is as follows:

生成设备身份与 MUA 会话材料
  ↓
获取 vfc_code 响应头 xy-ter-str
  → 解密 main_hmac，得到 key64
  ↓
发送 register/android
  → 服务端返回 GID
  ↓
发送 activate，携带临时 X25519 公钥
  → 解密 SSK，保存 sid
  ↓
请求 cold_start_config，取得 id_token
  ↓
生成内容请求的 MUA → canonical → SIG、S1
  ↓
用最终 path、query、白名单请求头和 body 计算 Shield
  ↓
发送请求

This is the client's call order, not a claim that every step has a cryptographic dependency. For example, the registration request does not use Shield and does not itself depend on key64. The fields used at each stage are as follows:

The activate response may also contain a token. In the client flow described here, even when a token already exists, cold_start_config is still requested to update it. sid and id_token enter Shield via xy-common-params; they are not independent fields in canonical.

Each response updates different state: registration saves data.g; activate decrypts main_ssk and saves data.session with the session. prefix as sid; cold_start_config reads the token from data.user_info.user_tags.id_token. Subsequent requests can use these new values only after the corresponding response has been processed successfully.

The same request headers and signature also cover the recommendation, search, and author-profile APIs. The interfaces that have been run end to end under an anonymous session include imagefeed and user/info and user/posted on edith.xiaohongshu.com, homefeed on rec.xiaohongshu.com, and search/notes on so.xiaohongshu.com. canonical uses only PATH, not the domain; across domains, only host and query change, while the signature formula remains the same.

The server has exhibited two types of session-level failure: code=-100, with msg set to 「Please log in」; and code=300011, with msg set to 「The current account is abnormal. Switch accounts and try again」. Both have occurred during consecutive requests. Retrying the same request on the same device usually still fails, while re-bootstrapping with a different device identity restores operation. These are expressions of server-side session and risk-control state; this article does not attribute them to an error in any particular signature input.

1. android_id and deviceId

deviceId uses the standard MD5 and UUID v3 version and variant-bit rules. The input is the UTF-8 bytes of the android_id text; for example, a 16-character hexadecimal string is still 16 bytes of ASCII and is not first decoded into 8 bytes.

h = MD5(utf8(android_id))
h[6] = (h[6] & 0x0F) | 0x30
h[8] = (h[8] & 0x3F) | 0x80
deviceId = 按 8-4-4-4-12 格式输出 h 的小写 hex

This hashes the name bytes directly without prepending a namespace, so it cannot be replaced with a UUID v3 API that takes an arbitrary namespace.

An equivalent formulation is (h[6] & 0x3F) | 0x30 and (h[8] & 0xBF) | 0x80. The bit 4 and bit 5 retained by the former are then both set to 1, while the bit 7 retained by the latter is also set to 1, so the final result is the same.

Example:

android_id = c91d22e4e041c6bd
MD5        = 4df7198147ee23390a7d7bbe597eca24
deviceId   = 4df71981-47ee-3339-8a7d-7bbe597eca24

Example:

import hashlib, uuid

def device_id(android_id: str) -> str:
    h = list(hashlib.md5(android_id.encode()).digest())
    h[6] = (h[6] & 0x3F) | 0x30
    h[8] = (h[8] & 0xBF) | 0x80
    return str(uuid.UUID(bytes=bytes(h)))

device_id("c91d22e4e041c6bd")
# '4df71981-47ee-3339-8a7d-7bbe597eca24'

2. x-mini-mua: session material and device profile

The MUA format is:

MUA = Base64URL(Part1_JSON_UTF8) + "." + Base64URL(Part2_ciphertext) + "."

No spaces are inserted during concatenation. Both Base64URL segments have their trailing = removed; the final period is retained. Part1 can be decoded directly as JSON, while Part2 must first be decrypted using the session shared secret.

Session initialization first obtains 64 bytes of random material s and then a 32-byte raw X25519 scalar. These two materials are reused within a session. The request counter and device runtime state may continue to change, so reusing the session material does not mean reusing the entire MUA.

2.1 X25519 and AES material

X25519 is computed over the prime field p = 2^255 - 19 . Before the scalar enters the curve operation, it is clamped:

scalar[0]  &= 0xF8
scalar[31]  = (scalar[31] & 0x7F) | 0x40

The raw scalar may be stored unchanged; clamping applies only to the copy supplied to the curve operation.

k     = X25519(scalar, basepoint9)
slotA = X25519(scalar, MUA_SERVER_PUBLIC_KEY)
key   = slotA[0:16]
iv    = slotA[16:32]

The base point 9 is encoded as a 32-byte little-endian value, namely 09 followed by 31 00The MUA server public key is:

c9b68adc9a607bd108c7b7bba0dd6b5a9eba716fb6027b52c861b838cae63637

SSK uses a different server public key; see Section 7. An all-zero shared secret must be treated as a failure.

the raw scalar, public key kand AES materials, respectively: multiplying the base point yields k, while X25519 with the fixed peer key yields slotA. Two sessions using different scalars both satisfy this relationship. Reconstructing Part2 with the derived key and IV rules out the possibility that the match applies only to the intermediate value from the preceding step.

native represents field elements using five 51-bit limbs. Their serialized form corresponds to the standard 32-byte little-endian output of X25519, so reproducing the protocol does not require retaining the internal limb representation.

When comparing native's intermediate buffer directly, each of the five limbs occupies one little-endian u64, for a total of 40 bytes. Let their values be h0…h4; the serialization formula is:

value = h0 + (h1 << 51) + (h2 << 102) + (h3 << 153) + (h4 << 204)
encoded = little_endian_32_bytes(value mod (2^255 - 19))

Here, the complete value is reduced modulo the relevant modulus rather than truncating each limb to 51 bits first; an intermediate limb may still contain an unnormalized carry. The same rule applies to k and slotA at this boundary.

Minimal implementation of the curve operations:

P = 2**255 - 19

def clamp(k: bytes) -> bytes:
    k = bytearray(k)
    k[0] &= 0xF8
    k[31] = (k[31] & 0x7F) | 0x40
    return bytes(k)

def x25519(scalar: bytes, u: bytes) -> bytes:
    k = int.from_bytes(clamp(scalar), 'little')
    x1 = int.from_bytes(u, 'little') & ((1 << 255) - 1)
    x2, z2, x3, z3, swap = 1, 0, x1, 1, 0
    for t in range(254, -1, -1):
        kt = (k >> t) & 1
        swap ^= kt
        if swap:
            x2, x3 = x3, x2
            z2, z3 = z3, z2
        swap = kt
        a = (x2 + z2) % P
        aa = (a * a) % P
        b = (x2 - z2) % P
        bb = (b * b) % P
        e = (aa - bb) % P
        c = (x3 + z3) % P
        d = (x3 - z3) % P
        da = (d * a) % P
        cb = (c * b) % P
        x3 = ((da + cb) ** 2) % P
        z3 = ((da - cb) ** 2 * x1) % P
        x2 = (aa * bb) % P
        z2 = (e * (aa + 121665 * e)) % P
    if swap:
        x2, x3 = x3, x2
        z2, z3 = z3, z2
    return ((x2 * pow(z2, P - 2, P)) % P).to_bytes(32, 'little')

Splitting the session material and slotA:

MUA_SERVER_PUBLIC_KEY = bytes.fromhex(
    'c9b68adc9a607bd108c7b7bba0dd6b5a9eba716fb6027b52c861b838cae63637')

pub = x25519(private_key, b'\x09' + b'\x00' * 31)   # Part1 的 k
slotA = x25519(private_key, MUA_SERVER_PUBLIC_KEY)
key, iv = slotA[:16], slotA[16:]                      # prefix16 / suffix16

2.2 Part1 fields

The key order in the current request construction is:

a, c, k, p, s, t, [u], v

Serialization uses compact JSON, leaves Unicode unescaped rather than converting it to ASCII escapes, and preserves field order. The registration path omits u. Historical full-replay samples also exist without t, so replays must preserve the original field set; adding a key changes Part1 and the complete MUA.

The body.c value for a registration, MUA.c, and the counter in the S1 frame all use the same value. The first registration in the cold-start client flow uses c=2, but the counter must be managed as session state rather than hard-coded as 2 for every request.

The Part1 field set varies by platform and version. On iOS 9.47.2, the observed Part1 uses a=ECFAAF02, and also includes i and m (a millisecond timestamp), n, and other fields not covered here; the request headers also include x-mini-nsig. Before reusing the formulas in this article across versions, verify the field set first.

Part1 construction:

part1 = {
    'a': 'ECFAAF01', 'c': counter, 'k': pub.hex(), 'p': 'a',
    's': random_material.hex(),
    't': {'c': 0, 'd': 0, 'f': 0, 's': 4098, 't': 0, 'tt': []},
    'v': '2.9.99',
}
part1_b64 = b64url(json.dumps(part1, separators=(',', ':')).encode())

For content requests with a non-empty OAID, insert u ('00000000' + oaid, between t and v), while the registration path omits u.

2.3 Part2 encoding and encryption

Part2's plaintext is a device-profile JSON object. The observed baseline contains 95 fields covering the package version, build information, installation time, battery, network, storage, process state, and more.

profile
  → 顶层键按字符串顺序排序
  → 紧凑 JSON，UTF-8
  → zlib，level=6，带 zlib 头
  → PKCS#7
  → AES-128-CBC(key, iv)
  → Base64URL，去掉 =

Key order is x0,x1,x10,…, not numeric order. Only the top-level keys are reordered here; nested objects are serialized in their existing order. The textual representation of floating-point numbers and the way Unicode is escaped are also part of the input bytes, so they must remain consistent across language implementations.

PKCS#7 padding rule:

pad = 16 - (len(compressed) mod 16)
padded = compressed || byte(pad) 重复 pad 次

Even when the compressed output is already aligned to 16 bytes, append 16 bytes, each containing the value 0x100x10.

For reverse verification, process the data in this order: Base64URL-decode, decrypt with CBC, verify and remove PKCS#7 padding, then decompress with zlib. The compressed data has a zlib wrapper and cannot be decompressed directly as raw DEFLATE. Identical JSON does not guarantee identical compressed bytes across compressors; reproducing the stream byte for byte also requires comparing the compression stream itself.

Field values must retain their JSON types. For example, x2 is an integer version code, x17 is an integer SDK value, x43 is a network string, and x242 is an empty array;"0" and 0 are not interchangeable either. x5's "JTdCJTdE" is the Base64 encoding of the text %7B%7D, not the Base64 encoding of {}. The profile here and the registration snapshot are two independent objects.

Validation at this layer should compare the plaintext JSON, zlib bytes, CBC ciphertext, and final MUA at the same time. Verifying only that data encrypted by your implementation can be decrypted by that same implementation does not prove that it matches native behavior. With fixed session material and the original profile, the complete MUA matches the native output byte for byte.

Reproducing the encryption and reproducing device data collection are two separate problems. A synthetic profile can validate the encoding pipeline, but it cannot establish the source of every field on a real Android device.

Outer pipeline:

import zlib
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

def pkcs7(data: bytes) -> bytes:
    n = 16 - len(data) % 16
    return data + bytes([n]) * n

def part2(profile: dict, key: bytes, iv: bytes) -> str:
    raw = json.dumps(
        {k: profile[k] for k in sorted(profile)},
        ensure_ascii=False, separators=(',', ':'),
    ).encode()
    cipher = Cipher(algorithms.AES(key), modes.CBC(iv)).encryptor()
    return b64url(cipher.update(pkcs7(zlib.compress(raw, level=6))) + cipher.finalize())

sorted(profile) is lexicographic string sorting, which directly produces x0, x1, x10, …the top-level key order; nested objects retain their input order.

2.4 Complete Mapping of the 95-Field Profile

The table below records the implemented profile construction. Package, build, and hardware fields come from device parameters; time and runtime-state fields can be supplied explicitly. Fixed values, time offsets, and seed-derived values are offline synthesis strategies, not formulas for collecting data from a real device.

Real-device confirmation (Xiaomi MI 6 / Lineage 15 / 9.43.1): the plaintext of MUA Part2 from the login process contains exactly these 95 keys, matching the synthesizer's key set. The package name, version, board, product, ABI, and build id/host/incremental all match the parameter table. In two later 95-key dumps from the same boot, only the timestamps, brightness, x258, x293, and x302/x303 changed. Observed values that were not used to change the synthesizer defaults: x70 was 25 rather than 3 on Wi-Fi; x258 could be 1; x235 was 3348494; and x302/x303 were nonempty and changed within the same process. No sample from a second vendor is available yet.

Let N be the current Unix time in milliseconds, I the saved installation time, and U max(1, uptime_ms)the saved uptime. If uptime is not provided, use N minus the saved boot time. Explicit runtime-state values take precedence over the defaults below; fixed device parameters must remain consistent within the same profile.

After expanding the merged entries, there are 95 keys in total. Existing slots may be overridden during construction, but adding or omitting keys will deviate from this baseline. The default key order for x234 is "1", "2", "3", with values I−67648, I−70281, and I−70435, respectively. x247 is:

{"0":53.333333333333336,"1":71.42857142857143,"2":71.42857142857143,"3":32.0,"4":85.71428571428571,"5":71.42857142857143}

2.5 Seed Derivation and Persistence Strategy

The device seed is used to regenerate the same synthetic identity. It is not an input to native key derivation and does not replace MUA's random s and scalar. A string seed is taken directly as UTF-8; a bytes seed is first converted to lowercase hex text and then encoded as UTF-8; when no seed is provided, a hex string representing 32 random bytes is generated and saved.

Define D(label,L) as the first L bytes of the following byte stream, with the counter j starting at 0:

D(label,L) = first_L_bytes(
    SHA256(seed_bytes || 00 || UTF8(label) || u32be(0))
 || SHA256(seed_bytes || 00 || UTF8(label) || u32be(1))
 || ...)

R(label,lo,hi) = lo + big_endian_u64(D(label,8)) mod (hi-lo+1)

The j here is the derived-stream counter and is unrelated to MUA.c. Persistent fields are initialized according to the following rules, with saved values taking precedence:

created_at_ms, installation time, boot time, and x146 are saved with the profile. Keeping only the seed while resetting created_at_ms each time changes the default installation and boot times; x146 itself does not depend on created_at_ms.

The same derivation rule is also used for the registered identity:

The synthetic installation directory is /data/app/~~{apk-dir}/com.xingin.xhs-{apk-id}. The synthetic filesystem state declares only item 12 successful, with fstype set to 61267; the other 11 items fail both checks. SoC serial and boot_id may both be omitted to replay the empty-file-tree baseline; there are no default fabricated values for MMC, UFS, or additional Settings.

The simulated JNI ID uses the full descriptor java/util/Map->方法签名 for Java String.hashCode: process h=(31*h+unit) mod 2^32 and then feed it into the MD5 described in Section 9.2. The simulated attestation uses SHA256(UTF8(deviceId)), while an empty identity is the SHA-256 of an empty byte string. These substitute values reproduce only the simulated environment; they are not the real formulas used by ART or the TEE.

The synthetic strategy keeps the attestation branch present, sets x148 to an empty string, and sets x128 to the missing-file text; x170 selects the observed branch based on whether the device parameter manufacturer equals Xiaomi . On an actual device, the branch is determined by class loading and collection results, so this selection strategy cannot replace real detection.

3. x-mini-sig: SHA-256 and a GF(2) Affine Transformation

digest = SHA256(utf8(canonical))
front  = A × bits(digest[0:16]) XOR b
sig    = hex(front || digest[16:32])

The output is 64 lowercase hexadecimal characters. The last 16 bytes are retained directly from the SHA-256 digest, while the first 16 bytes undergo a fixed GF(2) affine transformation.

3.1 Solving the Affine Relationship

Over GF(2), addition is XOR and multiplication is single-bit AND. Each output bit can be written as a linear combination of 128 input bits, XORed with a constant.

, first solve f(0), then evaluate e_j :

b      = f(0)
A[:,j] = f(e_j) XOR b

If only request-level inputs and outputs can be obtained, for each pair of digest first halves x and signature first halves y, construct:

[x 的 128 个 bit, 1] × coefficients = y 的某一 bit

Solve this using Gaussian elimination over GF(2). The augmented input matrix must have rank 129 to uniquely determine all coefficients; independent validation is also required beyond the training samples.

This is the method for recovering the affine transformation. The available materials retain the final matrix and independent captured-request vectors, but do not contain enough of the original solving records to reproduce all sampling and elimination steps from that time in this article. Eleven independent GET samples support the matrix's output consistency, but those 11 samples alone cannot recover 128×129 coefficients.

3.2 Bit Order and Evaluation

Write the matrix as 128 rows and 129 columns, with the final column storing the constant term:

v = int.from_bytes(digest[0:16], "big") | (1 << 128)

对输出行 ob = 0..127：
    bit = XOR(matrix[ob][j] AND ((v >> j) & 1), j=0..128)
    将 bit 写入输出字节 ob//8 的第 (7-ob%8) 位

Input column 0 corresponds to the least significant bit of the big-endian integer, whereas output row 0 corresponds to the highest bit of the result; the two directions differ. A compact representation of the complete matrix is provided in the appendix.

The intermediate result for one set of inputs is:

SHA256(canonical) =
31e2d1e8600bcfc46c0ba17ad2fdf6a9 34afff0c5c86368cb2f12521cafa5e7f

仿射变换后前 16 字节 =
839db883aafc9058c440fbaee6135968

SIG =
839db883aafc9058c440fbaee613596834afff0c5c86368cb2f12521cafa5e7f

POST uses the same formula; the difference lies in the body digest. Comparing signatures is meaningful only when canonical is exactly the same.

Combining canonical, the matrix, and the bit ordering:

import hashlib

def canonical(method: str, path: str, query: str, body: bytes, mua: str) -> bytes:
    return (f'{method}\n{path}\n{query}\n'
            f'{hashlib.sha256(body).hexdigest()}\n{mua}').encode()

def gf2_apply(digest16: bytes, rows) -> bytes:
    # rows：附录 A 的 128 行，每行 129 列（128 个输入位 + 1 个常量位）
    v = int.from_bytes(digest16, 'big') | (1 << 128)
    out = bytearray(16)
    for i, row in enumerate(rows):
        bit = 0
        for j, c in enumerate(row):
            if c:
                bit ^= (v >> j) & 1
        if bit:
            out[i // 8] |= 1 << (7 - i % 8)
    return bytes(out)

def compute_sig(method: str, path: str, query: str, body: bytes, mua: str, rows) -> str:
    d = hashlib.sha256(canonical(method, path, query, body, mua)).digest()
    return (gf2_apply(d[:16], rows) + d[16:]).hex()

rowsthe j column corresponds to input-string bit j (see the column order in Section 3.2), output i bit is written to byte i bit 7 - i % 8.

4. x-mini-s1: Digest Transformation, CBC, and Dual CRCs

S1 does not use SIG's output. It appends counter to canonical and recomputes SHA-256, then adds a timestamp to form the structure to be encrypted, finally outputting standard Base64.

When tracing this chain, the first observation was the 62-byte structure before Base64 encoding. Even with fixed random values, results could still vary across processes: one source of variation was the MUA, which had not yet been fully fixed in canonical, and another was the second-level timestamp written later. Only after fixing the complete canonical, counter, and time can the remaining differences be attributed to the algorithm.

Continuing to trace writes to the intermediate buffers, the result can be split into a 32-byte digest lane, a 44-byte seed, 48 bytes of ciphertext, and two CRCs. The variation that remained after fixing the complete canonical was ultimately traced to the return value oftime(NULL); after fixing this second-level value, the output was consistent across processes.

The lane was initially modeled with a fixed XOR-affine model, but new samples contradicted it. The four transformations were then recovered separately from the VM instructions and memory writes. For CBC, the lookup-table layer was decomposed into the AES S-box and input/output XOR masks, yielding the explicit round function in Section 4.4. These masks are constants recovered from the table, not round keys derived by running the standard AES key schedule on K0.

4.1 SHA-256 and Word Reordering

nested = utf8(canonical) || u32le(counter)
digest = SHA256(nested)
state  = digest 的 4 字节块按 (7,0,2,5,6,3,1,4) 重排

The byte order within each 4-byte block is not reversed.

counter is encoded as an unsigned 32-bit integer and must match MUA.c. It participates in the nested digest and also appears in the final frame header; changing only the frame header cannot produce a complete S1 for the corresponding new counter.

4.2 Four-Part Lane Transformation

The state is divided into four 8-byte parts, which are processed separately and then concatenated.

Use reflected IEEE CRC-32, with polynomial 0xEDB88320 and initial value 0xFFFFFFFF:

crc32_internal(data) = CRC 循环结束时的寄存器值
crc32(data)          = crc32_internal(data) XOR 0xFFFFFFFF
swap(b)             = (b >> 4) | ((b & 0x0F) << 4)

The first part first swaps the high and low nibbles of each byte, then uses the low 8 bits of the internal CRC as a uniform XOR mask:

a[i]   = swap(input[i])
mask   = crc32_internal(a) & 0xFF
out[i] = a[i] XOR mask

The second part:

mix = ((input[0]<<4) | (input[7]>>4))
      XOR ((input[6]<<4) | (input[1]>>4))
a[i] = input[i] XOR (mix & 0xFF)

out[i] = a[i]       当 i 为 3 或 5
       = swap(a[i]) 其它位置

The third part uses a nibble S-box (indices 0..15):

B 2 F 6 8 9 E 3 1 D 7 5 C A 0 4

substituted = 对每个字节，依次追加 SBOX[高4位] 和 SBOX[低4位]
              # 每个结果各占一个字节，值为 0..15，共 16 字节
mask = crc32_internal(substituted) & 0xFF
out[i] = input[i] XOR mask

The CRC input is the expanded 16 bytes; the final XOR is still applied to the original 8 bytes.

The fourth part:

C = 0xAE
first = input[0] XOR C
first_swapped = swap(first)
second = input[1] XOR first_swapped XOR C
second_swapped = swap(second)
mask = first_swapped XOR second_swapped XOR C

out[0] = first_swapped XOR second_swapped XOR first
out[1] = second_swapped XOR second
out[2:8] = input[2:8] XOR mask

The four outputs are concatenated into lane32. The CRC recurrence for the first part was compared byte by byte with the native intermediate state; the other parts use independent transformations, so the first-part formula cannot be applied to all 32 bytes.

4.3 Seed and Frame Structure

seed44   = u32le(timestamp_seconds) || 51 24 4a 19 || 00 00
           || lane32 || 17 0e
crc_seed = u32be(crc32(seed44))
plain48  = seed44 || 04 04 04 04
body48   = fused_aes_cbc(plain48)

source53 = 01 || body48 || crc_seed
crc_src  = u32be(crc32(source53))
raw62    = 00 || u32le(counter) || source53 || crc_src
S1       = Base64(raw62)

The second CRC covers 53 bytes and excludes the outermost marker and counter. If it is also appended to the temporary source buffer, that buffer is 57 bytes long; this is distinct from the CRC input length.

The 62 bytes become 84 characters in standard Base64, with one padding character at the end=.

timestamp_seconds is the Unix time in seconds, written as a u32. It is different from the uptime in milliseconds in Part1.t.t and does not participate in nested SHA-256; changes in time appear in the seed, CBC ciphertext, and subsequent verification layers.

4.4 Fused AES-like CBC

CBC IV:

6e2052fb01f05fd238ad54534bd00831

First XOR the current plaintext block with previous, whose initial value is IV. The block transform uses the standard AES S-box and MixColumns, but the operation order and round-key table are organized as follows:

state = ShiftRows(Transpose(plain_block XOR previous))
for Ki in K0..K8:
    state = SBOX[state XOR Ki]
    state = ShiftRows(MixColumns(state))
state = SBOX[state XOR FA] XOR C
cipher_block = Transpose(state)
previous = cipher_block

Transpose converts the 16-byte column-major input into a row-major state; ShiftRows rotates row r left by r positions. Round-key bytes correspond one-to-one to the row-major state above, so the key must not be transposed again.

The nine round keys, in order, are:

786873306f6832697534616e376f6732
5a3241711e764471e0d4fb95096b0c3e
43713041275115398f5bfa6ff3c8c4fa
1e6f5f1e27766300174cf798864b8f75
ed82ddc30e781b293c70b32b4539b6c3
52d00dce760e1578cabaddf62717a162
2bfbf638faf4e18c2d9711e7789c3d5f
93689ea6fe0aeb0467f0ad4a819ba6f9
fa920caa767c97888c7ca1eb61bf19e0

FA = 69fbf75dee92059829554ea58b130aea
C  = 34cf3865648a181d73d6ffaa5f55bf34

K0 corresponds to ASCII xhs0oh2iu4an7og2. The final layer contains only one S-box and no MixColumns.

The AES S-box is constructed by first finding the multiplicative inverse in GF(2^8), followed by an affine transformation. The irreducible polynomial is 0x11B, with 0 treated as its own inverse:

SBOX[x] = inv(x) XOR rotl8(inv(x),1) XOR rotl8(inv(x),2)
          XOR rotl8(inv(x),3) XOR rotl8(inv(x),4) XOR 0x63

MixColumns computes the following for each column [a,b,c,d]:

2a XOR 3b XOR c XOR d
a XOR 2b XOR 3c XOR d
a XOR b XOR 2c XOR 3d
3a XOR b XOR c XOR 2d

Multiplication is performed in the same GF(2^8). Padding the 44-byte seed with four 0x04 makes exactly three blocks. The transformation above cannot be replaced directly with standard AES-CBC using an initial key.

There are currently 23 fixed-time native full-frame comparisons with counter=1, and 5 full-frame comparisons with counter=2 at fixed times. The latter cover five distinct Unix-second values, including the same second as the counter=1 corpus, an adjacent second, a one-hour offset, and 1 and 1700000000. These five raw62 values match the formula byte for byte with the same canonical, counter=2, and corresponding timestamp. 51244a19 and 170e remain unchanged in these comparisons.

A separate arm64 libtiny from Android 9.47.0 (build 9470809) had contents identical byte for byte to the 9.43.1 sample. Therefore, the nine round keys, FA, C, 51244a19 and 170e did not change between these two application versions. These constants do not appear as contiguous plaintext in the SO; the determination of drift is based on the complete library images, not on literal-value searches.

To turn the formulas above into an executable check, the following is a minimal implementation with no third-party dependencies;SBOX is generated directly by finding the multiplicative inverse in GF(2^8), matching the standard AES S-box:

import hashlib, struct

def gf_mul(a, b):
    r = 0
    for _ in range(8):
        if b & 1: r ^= a
        a = ((a << 1) ^ 0x11B) & 0xFF if a & 0x80 else (a << 1) & 0xFF
        b >>= 1
    return r

def gf_inv(x):
    if x == 0: return 0
    for y in range(1, 256):
        if gf_mul(x, y) == 1: return y

def rotl(v, n): return ((v << n) | (v >> (8 - n))) & 0xFF

SBOX = bytes(gf_inv(x) ^ rotl(gf_inv(x), 1) ^ rotl(gf_inv(x), 2)
             ^ rotl(gf_inv(x), 3) ^ rotl(gf_inv(x), 4) ^ 0x63 for x in range(256))

def crc_state(data):
    s = 0xFFFFFFFF
    for v in data:
        s ^= v
        for _ in range(8):
            s = (s >> 1) ^ 0xEDB88320 if s & 1 else s >> 1
    return s

def crc32(data): return crc_state(data) ^ 0xFFFFFFFF

def ns(data): return bytes((v >> 4) | ((v & 0x0F) << 4) for v in data)

WORD_ORDER = (7, 0, 2, 5, 6, 3, 1, 4)
LANE3 = (0xB, 0x2, 0xF, 0x6, 0x8, 0x9, 0xE, 0x3, 0x1, 0xD, 0x7, 0x5, 0xC, 0xA, 0x0, 0x4)
ROUND_KEYS = tuple(bytes.fromhex(h) for h in (
    "786873306f6832697534616e376f6732", "5a3241711e764471e0d4fb95096b0c3e",
    "43713041275115398f5bfa6ff3c8c4fa", "1e6f5f1e27766300174cf798864b8f75",
    "ed82ddc30e781b293c70b32b4539b6c3", "52d00dce760e1578cabaddf62717a162",
    "2bfbf638faf4e18c2d9711e7789c3d5f", "93689ea6fe0aeb0467f0ad4a819ba6f9",
    "fa920caa767c97888c7ca1eb61bf19e0"))
FINAL_A = bytes.fromhex("69fbf75dee92059829554ea58b130aea")
FINAL_C = bytes.fromhex("34cf3865648a181d73d6ffaa5f55bf34")
CBC_IV = bytes.fromhex("6e2052fb01f05fd238ad54534bd00831")

def digest_lane(canonical: bytes, counter: int) -> bytes:
    digest = hashlib.sha256(canonical + struct.pack('<I', counter)).digest()
    state = b''.join(digest[i*4:i*4+4] for i in WORD_ORDER)
    b0, b1, b2, b3 = state[0:8], state[8:16], state[16:24], state[24:32]
    s = ns(b0)
    lane0 = bytes(v ^ (crc_state(s) & 0xFF) for v in s)
    cn = ((b1[0] << 4) | (b1[7] >> 4)) ^ ((b1[6] << 4) | (b1[1] >> 4))
    lane1 = bytes(v if i in (3, 5) else ns(bytes((v,)))[0]
                  for i, v in enumerate(bytes(v ^ (cn & 0xFF) for v in b1)))
    sub = bytes(t for v in b2 for t in (LANE3[v >> 4], LANE3[v & 0x0F]))
    lane2 = bytes(v ^ (crc_state(sub) & 0xFF) for v in b2)
    first = b3[0] ^ 0xAE
    c0 = ns(bytes((first,)))[0]
    second = b3[1] ^ c0 ^ 0xAE
    c1 = ns(bytes((second,)))[0]
    mask = c0 ^ c1 ^ 0xAE
    lane3 = bytes((c0 ^ c1 ^ first, c1 ^ second, *(v ^ mask for v in b3[2:])))
    return lane0 + lane1 + lane2 + lane3

def transpose(s): return bytes(s[r*4+c] for c in range(4) for r in range(4))

def shift_rows(s): return b''.join(s[r*4+r:r*4+4] + s[r*4:r*4+r] for r in range(4))

def mix_columns(s):
    t = transpose(s); out = bytearray()
    for off in range(0, 16, 4):
        a, b, c, d = t[off:off+4]
        out += bytes((gf_mul(a,2)^gf_mul(b,3)^c^d, a^gf_mul(b,2)^gf_mul(c,3)^d,
                      a^b^gf_mul(c,2)^gf_mul(d,3), gf_mul(a,3)^b^c^gf_mul(d,2)))
    return transpose(bytes(out))

def s1_block(block):
    s = shift_rows(transpose(block))
    for rk in ROUND_KEYS:
        s = bytes(SBOX[v ^ m] for v, m in zip(s, rk))
        s = shift_rows(mix_columns(s))
    s = bytes(SBOX[v ^ a] ^ c for v, a, c in zip(s, FINAL_A, FINAL_C))
    return transpose(s)

def s1_cbc(payload):
    prev, out = CBC_IV, bytearray()
    for off in range(0, len(payload), 16):
        prev = s1_block(bytes(x ^ y for x, y in zip(payload[off:off+16], prev)))
        out += prev
    return bytes(out)

def build_s1_raw(canonical: bytes, counter: int, timestamp_s: int) -> bytes:
    lane = digest_lane(canonical, counter)
    seed = (struct.pack('<I', timestamp_s) + bytes.fromhex('51244a19')
            + b'\x00\x00' + lane + bytes.fromhex('170e'))
    source = b'\x01' + s1_cbc(seed + b'\x04' * 4) + crc32(seed).to_bytes(4, 'big')
    return (b'\x00' + struct.pack('<I', counter) + source
            + crc32(source).to_bytes(4, 'big'))

5. main_hmac: Recovering key64 from Shield

main_hmac is encrypted material returned by the server, not the HMAC result for the current request. The client obtains a Base64 string from/api/sns/v1/system_service/vfc_codethe response headerxy-ter-str.

raw96 = Base64Decode(xy-ter-str)
plain = CustomAES-CBC-Decrypt(raw96, round_keys, IV=0)
plain = header16 || key64 || footer16

Fixed header:

3501323400020861667a666607176639

The decrypted result must be 96 bytes, with the first 16 bytes equal to header. Shield uses the middle 64 bytes; the footer does not participate in the request HMAC. No PKCS#7 unpadding is performed here.

Ninety-six bytes are exactly six AES blocks, but the length alone cannot prove that this is AES. Other evidence includes the standard S-box, row shifts, and column mixing in the block transform, along with a deviceId-driven round-key expansion. The difference from standard AES is confined to the Rcon values below, not to a different custom S-box.

5.1 Custom AES Key Expansion

deviceId uses a 36-character hyphenated UUID in ASCII:

key16 = utf8(deviceId)[0:16] XOR MASK
MASK  = f1892131ff001123f1001356f1234890

It then performs AES-128 key expansion, replacing Rcon with the following 10 big-endian u32 values:

12310000 02000100 04020000 08020200 10102000
30020400 40002000 80002000 1b002000 36200200

w[0..3] = key16 按大端拆成四个 u32
for i = 4..43:
    t = w[i-1]
    if i mod 4 == 0:
        t = SubWord(RotWord(t)) XOR Rcon[i/4-1]
    w[i] = w[i-4] XOR t

RotWord rotates four bytes left by one byte, and SubWord applies the standard AES S-box to each byte. This produces 44 words, or 176 bytes of round keys.

During CBC decryption, P_i = D_round_keys(C_i) XOR C_(i-1)the predecessor of the first block is 16 zero bytes. Plaintext slices [0:16],[16:80], and [80:96] yield the header, key64, and footer, respectively.

The block-cipher round function is standard AES: initial AddRoundKey, nine rounds of SubBytes, ShiftRows, MixColumns, and AddRoundKey, followed by a final round that omits MixColumns. What changes is the key schedule; calling a standard AES library with only key16 produces a different result.

Example:

deviceId = aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee
key16    = 90e840509e617042dc627134930e7cf3
w[4]     = 29c94d8c

To establish that this recovery is correct, check the plaintext header and key64 across deviceIds. The decrypted results for 18 sets of corresponding materials are consistent; an encrypt/decrypt round trip alone cannot replace this comparison.

There is no local formula that can derive key64 offline from deviceId alone. libxyass writes xy-ter-str to prefs s/main_hmac only after receiving it, then uses deviceId as the AES wrapping key to decrypt the 64-byte value; there is no f_key literal in the .so. A cold start with empty prefs retains the 70-byte Shield form. Under the normal protocol, key64 can be recovered once main_hmac has been obtained.

5.2 Reverse Encryption

Given the same deviceId, known key64, and footer, main_hmac can be reconstructed in reverse:

plain96 = header16 || key64 || footer16
C_prev = 16 个零字节
for 每个连续 16 字节明文块 P_i:
    C_i = E_round_keys(P_i XOR C_prev)
    C_prev = C_i
main_hmac_b64 = Base64(C_0 || C_1 || ... || C_5)

E uses the custom round keys from Section 5.1. The plaintext consists of exactly six blocks, with no additional padding. The footer must be retained as input; it cannot be omitted simply because Shield does not use it. Reverse encryption only demonstrates that the wrapper can be reconstructed; it does not mean that an unknown key64 can be generated from deviceId or that new material matching the server-issued material can be produced independently.

key64 can likewise be extracted using only the standard library.SBOX is the same as the example in Section 4.4; decryption uses the standard AES block function with the round keys from this section:

import base64, hashlib

RCON = [0x12310000, 0x02000100, 0x04020000, 0x08020200, 0x10102000,
        0x30020400, 0x40002000, 0x80002000, 0x1b002000, 0x36200200]
MASK = bytes.fromhex('f1892131ff001123f1001356f1234890')
HEADER = bytes.fromhex('3501323400020861667a666607176639')

def rotw(w): return ((w << 8) | (w >> 24)) & 0xFFFFFFFF

def subw(w): return int.from_bytes(bytes(SBOX[b] for b in w.to_bytes(4, 'big')), 'big')

def key_expand(key16: bytes) -> bytes:
    w = [int.from_bytes(key16[i*4:i*4+4], 'big') for i in range(4)]
    for i in range(10):
        t = subw(rotw(w[-1]))
        w.append(w[-4] ^ t ^ RCON[i])
        for _ in range(3):
            w.append(w[-4] ^ w[-1])
    return b''.join(x.to_bytes(4, 'big') for x in w)

def xtime(a): a <<= 1; return (a ^ 0x11B) & 0xFF if a & 0x100 else a & 0xFF

def gmul(a, b):
    r = 0
    for _ in range(8):
        if b & 1: r ^= a
        b >>= 1; a = xtime(a)
    return r

INV = [0] * 256
for i, v in enumerate(SBOX):
    INV[v] = i

def inv_shift_rows(s):
    return bytes([s[0], s[13], s[10], s[7], s[4], s[1], s[14], s[11],
                  s[8], s[5], s[2], s[15], s[12], s[9], s[6], s[3]])

def inv_mix_columns(s):
    o = bytearray(16)
    for c in range(4):
        a = s[c*4:c*4+4]
        o[c*4+0] = gmul(a[0],14) ^ gmul(a[1],11) ^ gmul(a[2],13) ^ gmul(a[3],9)
        o[c*4+1] = gmul(a[0],9)  ^ gmul(a[1],14) ^ gmul(a[2],11) ^ gmul(a[3],13)
        o[c*4+2] = gmul(a[0],13) ^ gmul(a[1],9)  ^ gmul(a[2],14) ^ gmul(a[3],11)
        o[c*4+3] = gmul(a[0],11) ^ gmul(a[1],13) ^ gmul(a[2],9)  ^ gmul(a[3],14)
    return bytes(o)

def xor16(a, b): return bytes(x ^ y for x, y in zip(a, b))

def aes_decrypt_block(block: bytes, rk: bytes) -> bytes:
    s = xor16(block, rk[160:176])
    for i in range(9, 0, -1):
        s = inv_shift_rows(s)
        s = bytes(INV[b] for b in s)
        s = xor16(s, rk[i*16:(i+1)*16])
        s = inv_mix_columns(s)
    s = inv_shift_rows(s)
    s = bytes(INV[b] for b in s)
    return xor16(s, rk[0:16])

def main_hmac_key64(main_hmac_b64: str, device_id: str) -> bytes:
    raw = base64.b64decode(main_hmac_b64)
    key16 = bytes(a ^ b for a, b in zip(device_id.encode()[:16], MASK))
    rk = key_expand(key16)
    plain, prev = bytearray(), bytes(16)
    for off in range(0, len(raw), 16):
        block = raw[off:off+16]
        plain += xor16(aes_decrypt_block(block, rk), prev)
        prev = block
    assert bytes(plain[:16]) == HEADER
    return bytes(plain[16:80])   # key64

6. Shield: Request HMAC and SSK Extension

The outer format is:

Shield = "XY" + Base64(raw)

Standard Base64 is used, with padding preserved. raw has two possible lengths: 70 bytes for the basic structure, and 118 bytes when both main_hmac and main_ssk are present.

6.1 HMAC Input

Concatenate the components in the following order, with no line breaks, question marks, or extra separators:

source =
    path
 || query
 || xy-common-params
 || xy-direction
 || xy-live-net-bandwidth
 || xy-live-net-networktype
 || "platform=android&build=9431801&deviceId=" || deviceId
 || xy-scene
 || body

Missing headers are treated as empty strings, and the result is then encoded as UTF-8 bytes. host, scheme, and METHOD are not part of this input. The platform segment is rebuilt from version constants and deviceId rather than read directly from the potentially differentxy-platform-info value.

body must match the content actually sent. For form requests, sign the already percent-encoded form text; for JSON, sign the final serialized bytes rather than regenerating a semantically equivalent JSON object.

6.2 Custom HMAC-MD5

The outer HMAC structure is unchanged:

inner = MD5_mod((key64 XOR 0x36) || source)
outer = MD5_mod((key64 XOR 0x5c) || inner)
request_hmac = outer

key64 is already exactly 64 bytes; XOR 0x36the inner and outer pads XOR 0x5care formed by XORing each byte with the same constant. inner and outer are both 16-byte raw digests; when computing outer, do not convert inner to hex text.

The native initialization and 64-round compression still retain the MD5 structure, so the initial state, rotation table, additive constants, message indices, and register updates can be compared item by item. The modifications listed below collectively form MD5_mod.

The initial state is the reverse of standard MD5:

a = 0x10325476
b = 0x98badcfe
c = 0xefcdab89
d = 0x67452301

T table:

e9c9b756 d71ba479 241081db 681088d9 9b14f7af ff1f5bb1 881cd7be 66666122
f6666193 a619639e 49140921 c11dceee f51c0faf 4717c62a a9104613 fd169501
f61e2562 02741453 d221e691 e213fbc9 2261cde6 f2d50d97 425a14ed a277e905
f277a3f9 626f12d9 922a4c9a c040b340 265e5a51 e9f6c7aa d63f105d c35707d6
fffc3942 977cd691 a4bcea44 4bdccfa9 bebcbc70 288c7ec6 f6cc4b60 eaac27fa
d4ec1095 d9dcd039 e6dc88e5 048c1d05 1fa27cf9 6d9d6122 c4ac5665 fde5391c
f4292244 ab9423a7 f593a039 655b59c3 452aff97 f5ef247d 85845dd1 850ccc92
f99926e0 f9997e4f a9994314 c5537e82 450811a1 450811a6 bd3af235 eb86d391

First copy T to K, then modify:

K[16] = T[16] & 0xff00ff00
K[19] = T[19] & 0xff0011ff
K[29] = T[29] & 0xff110010
交换 K[39]、K[40]
交换 K[41]、K[42]

The rotation counts initially use the four standard MD5 groups, each repeated four times:

7,12,17,22 / 5,9,14,20 / 4,11,16,23 / 6,10,15,21

Then overwrite 10 positions:

S[0]=6, S[1]=13, S[3]=21, S[7]=20, S[10]=16, S[13]=13
S[39]=4, S[40]=23, S[41]=16, S[42]=11

Round functions and message indices:

i < 16: f(b,c,d) = (b&c) | (~b&d), g=i
i < 32: f(b,c,d) = (d&b) | (~d&c), g=(5*i+1)%16
i < 48: f(b,c,d) = b^c^d,           g=(3*i+5)%16
其它:   f(b,c,d) = c^(b|~d),       g=(7*i)%16

覆盖 g[39],g[40],g[41],g[42] 为 13,10,3,0

Starting at round 39, reorder the input registers according to PIN:

39:       (3,0,1,2)
40..42:   (0,1,2,3)
43:       (1,2,0,3)
44:       (1,3,0,2)
45..63:   (2,3,0,1)

Split each 64-byte block into 16 little-endian u32 values, denoted M:

a,b,c,d = state
for i = 0..63:
    按轮号选择 f 和 g
    A,B,C,D = (a,b,c,d)                   当 i <= 38
            = 按 PIN[i] 重排 (a,b,c,d)   其它情况

    x = (A + f(B,C,D) + M[g] + K[i]) & 0xffffffff
    new = (B + rotl32(x,S[i])) & 0xffffffff

    a,b,c,d = D,new,B,C    当 i <= 38 或 i == 63
            = B,C,D,new    当 39 <= i <= 62

state = 原 state 与 (a,b,c,d) 逐字相加，截断到 32 位

Padding follows the MD5 rules: append 0x80, pad with zeros until the length is congruent to 56 modulo 64, then append the original bit length as a little-endian u64. After all blocks have been processed, output the state as four little-endian u32 values.

For this kind of variant, the initial state, table constants, round functions, message indices, and register-update order must each be confirmed. Replacing only MD5's T table is insufficient; the 32-bit truncation before rotation must not be omitted either.

One digest check value:

MD5_mod(ASCII("00000000-0000-0000-0000-000000000000"))
= 94d3c51d13033535c7ca0a1183028c1d

6.3 70-byte base structure

The constants are as follows:

RC4_KEY = UTF-8("逆向违法，即刻停止！Reverse engineering is illegal. Cease now!")
MASK36  = 606b39fd3c39443008502297d736827cf3903c05d0e77e0737f77066f77460435bd69068
MASK16  = b3661b2425c8d1cf9674e8218b30b91a
HDR_PREFIX = 8dfe00055569
HDR_SUFFIX = a6d7a54f949cf960
EMPTY_HF   = 9a89
NONEMPTY_HF = 9e8a

RC4_KEY is 72 bytes of UTF-8. Use the standard RC4 KSA and PRGA:

tail = request_hmac XOR MASK16     有 main_hmac 时
     = MASK16                     无 main_hmac 时

plain54 = c1 || (utf8(deviceId) XOR MASK36) || f4 || tail
header16 = HDR_PREFIX || hf || HDR_SUFFIX
raw70 = header16 || RC4(RC4_KEY, plain54)

hf selects EMPTY_HF or NONEMPTY_HF based on whether main_hmac exists; it is not a truncated HMAC value. The base structure is 16 + 1 + 36 + 1 + 16 = 70 bytes long.

When main_hmac is absent, the request HMAC is not computed, so raw70 depends only on deviceId; changing path, query, or body does not change it. Only after main_hmac is obtained does tail bind the request contents to Shield. 70B and 118B are encoded as 96 and 160 Base64 characters, respectively; with XY added, their total lengths are 98 and 162 characters.

6.4 48-byte SSK extension

Comparing the outputs with and without SSK shows that the base portion remains 70 bytes, with an additional 48-byte tail. Further splitting the data before serialization reveals two TLVs: a 20-byte proof with tag 4 and a 24-byte proof with tag 5.

native uses the standard Base64 ASCII representation of main_ssk, rather than using its 32-byte raw value directly:

main_ssk_b64 = Base64(main_ssk32)             # 44 字符，含 =
ssk_proof24  = nonce4 || SHA1(ASCII(main_ssk_b64) || nonce4)
req_proof20  = SHA1(request_hmac16 || ssk_proof24)
plain48      = 04 14 || req_proof20 || 05 18 || ssk_proof24

The length bytes 14, 18 are hexadecimal and represent 20 and 24, respectively.

The extension uses the specified range of the RC4 keystream:

stream = RC4(RC4_KEY, 116 个零字节)
extension = plain48 XOR stream[68:116]
raw118 = raw70 || extension

The offset is 68 rather than 70 because the first two bytes of the final raw value are inserted during post-processing. Keep the equivalent construction of the base structure separate from the stream offset of the extension; do not concatenate plain54 and plain48 and run RC4 over the result again.

When main_hmac is empty, the extension branch is not entered even if SSK is present. Changing query changes request_hmac and then req_proof; changing SSK changes ssk_proof first.

The nine fixed-nonce samples covering three SSKs and three URLs all reproduce the complete 118 bytes. The 4-byte nonce comes from the generator in the libxyass extension branch; it is not getrandom or /dev/urandom. The function increments a process-local atomic counter, then XORs it with the low 32 bits of the address of the sp+0x20 slot on the current stack, writing the result in little-endian order:

counter = atomic_inc(global_counter)          # 观测到的会话从 1 开始
nonce4  = u32le(counter XOR trunc32(sp + 0x20))

It then uses SHA1(ASCII(main_ssk_b64) || nonce4) to fill tag 5. sp+0x20 is part of the runtime stack layout, not a protocol constant; because stack addresses are stable in the emulator, the low bits of the nonce change with the counter across consecutive requests and flip via XOR when the counter crosses a power of 16, rather than increasing monotonically. ASLR on physical devices makes this a different 4-byte value for each process. A portable implementation that does not know the stack address should still treat the nonce as a per-request input rather than hard-coding a cross-process constant table. Thirty-two consecutive native outputs from the same process satisfy the formula above.

The 70B form without main_hmac can be reproduced independently without a custom MD5:

import base64
from urllib.parse import urlsplit

RC4_KEY = '逆向违法，即刻停止！Reverse engineering is illegal. Cease now!'.encode()
MASK36 = bytes.fromhex('606b39fd3c39443008502297d736827cf3903c05d0e77e0737f77066f77460435bd69068')
MASK16 = bytes.fromhex('b3661b2425c8d1cf9674e8218b30b91a')
HDR_PREFIX = bytes.fromhex('8dfe00055569')
HDR_SUFFIX = bytes.fromhex('a6d7a54f949cf960')
EMPTY_HF = bytes.fromhex('9a89')

def rc4(key: bytes, data: bytes) -> bytes:
    s = list(range(256)); j = 0
    for i in range(256):
        j = (j + s[i] + key[i % len(key)]) & 0xFF
        s[i], s[j] = s[j], s[i]
    i = j = 0
    out = bytearray()
    for b in data:
        i = (i + 1) & 0xFF
        j = (j + s[i]) & 0xFF
        s[i], s[j] = s[j], s[i]
        out.append(b ^ s[(s[i] + s[j]) & 0xFF])
    return bytes(out)

def shield_70b(device_id: str, url: str, headers: dict) -> str:
    parts = urlsplit(url)
    devid = device_id.encode()
    plain = bytearray([0xC1])
    plain += bytes(a ^ b for a, b in zip(devid, MASK36))
    plain.append(0xF4)
    plain += MASK16
    payload = HDR_PREFIX + EMPTY_HF + HDR_SUFFIX + rc4(RC4_KEY, bytes(plain))
    return 'XY' + base64.b64encode(payload).decode()

shield_70b('4ee6f1f8-75e5-313d-993b-e5a8aaedbaae',
           'https://edith.xiaohongshu.com/api/sns/v1/system_service/vfc_code'
           '?phone=13800138000&zone=1&type=login',
           {'xy-common-params': '', 'xy-direction': '90'})
# 'XYjf4ABVVpmomm16VPlJz5YE0FodlR1qnMmVnCszuzcoCJfJOhpEN1aN/UnJcBJa1K0LUqi0v+Qdey0hyMHeAio6PLpd74zQ=='

7. SSK: X25519 and AES-256-GCM

The client sends an ephemeral X25519 public key to /api/sns/v1/user/activate under the field name client_public_key_base64. The public key is first Base64-encoded, then percent-encoded according to form rules.

The SSK server public key is:

2abd22ca0b02bb78a760935708be24e09bb369843669f235c5bc1cec11fab26f

shared = X25519(client_private, SSK_SERVER_PUBLIC)
blob   = Base64Decode(data.ssk)
       = iv12 || ciphertext32 || tag16
ssk32  = AES-256-GCM-Decrypt(shared, iv12, ciphertext32 || tag16, AAD=空)

The decoding layer can also accept URL-safe Base64. The shared secret is used directly as a 256-bit AES key, with no HKDF in between; an all-zero shared secret must be rejected, the GCM tag must be verified, and the plaintext must be 32 bytes. The SSK GCM nonce is 12 bytes and is unrelated to nonce4 in the Shield extension.

This process must not be mixed up with the MUA key/IV split: MUA splits the shared secret into an AES-128-CBC key and IV, whereas SSK uses the complete shared secret for AES-256-GCM and reads the IV from the response.

The request and response in this exchange must use the same ephemeral key pair: send the public key with the request and retain the corresponding private key while processing the response. Generating a new private key and then decrypting the original response changes the shared secret and causes GCM tag verification to fail. The MUA scalar and activate's ephemeral private key should likewise be managed as separate session materials.

RFC 7748 vectors can validate the standard X25519 primitive, and a constructed GCM message can validate decryption and tag-rejection behavior, but neither alone proves that a particular real activate response used the same materials.

Deriving main_ssk:

from cryptography.hazmat.primitives.ciphers.aead import AESGCM

blob = base64.b64decode(encrypted_ssk)   # iv[12] || ciphertext+tag
main_ssk = AESGCM(shared).decrypt(blob[:12], blob[12:], None)   # 32B

shared using the x25519(), with this section's SSK server public key as the peer rather than MUA's MUA_SERVER_PUBLIC_KEY. The resulting 32 bytes are used directly as the AES-256-GCM key; do not reuse slotA from Section 2.1.

8. Registration request: GID and encrypted snapshot

The registration endpoint is POST /api/v1/register/android. The request carries MUA, SIG, and S1, but not Shield. The body is compact JSON generated with the following key order:

a, c, d, e, k, p, s, v

a, c, k, p, s, and v are consistent with those in MUA Part1 from the same session. e is the plaintext identity object, which in the empty-identity sample from the first launch is:

{"device_id":"","rr":"0","sid":"","uid":""}

The encoding process for d is:

snapshot JSON
  → 顶层键按字符串排序、紧凑 UTF-8
  → zlib(level=6)
  → PKCS#7
  → AES-128-CBC(slotA[0:16], slotA[16:32])
  → Base64URL，去掉 =

d shares slotA and the outer algorithm with MUA Part2 from the same session, but its plaintext is a different data structure. MUA Part2's string cannot be used directly as d.

In one set of registration samples, the snapshot plaintext was 960 bytes, the zlib output was 448 bytes, and the padded ciphertext was 464 bytes. The length is determined by the compressed content; it is not a fixed protocol value.

The dependency order for the registration signature is:

会话材料 → MUA
同一 slotA → 加密 snapshot 得到 d → 生成最终注册 body
MUA + 最终 body 的 SHA256 → canonical → SIG / S1

Once signing is complete, the body is not reordered or modified. The server response's data.g was 56 hex characters in the observed samples, and was subsequently written unchanged to x-mini-gid and gid in the common parameters.

The field d and body assembly:

from collections import OrderedDict

def register_body(snapshot: dict, *, counter: int, pub_hex: str,
                  random_hex: str, key: bytes, iv: bytes) -> bytes:
    d = part2(snapshot, key, iv)   # 与 Part2 同一外层：zlib -> PKCS7 -> AES-CBC -> Base64URL
    body = OrderedDict((
        ('a', 'ECFAAF01'), ('c', counter), ('d', d),
        ('e', OrderedDict((('device_id', ''), ('rr', '0'), ('sid', ''), ('uid', '')))),
        ('k', pub_hex), ('p', 'a'), ('s', random_hex), ('v', '2.9.99'),
    ))
    return json.dumps(body, separators=(',', ':')).encode()

Here, key and iv share slotA with MUA Part2 from the same registration session; after assembling the body, calculate SIG and S1 in the order specified in section 10.

9. Hash and encoding fields in snapshot

The snapshot fields are subject to conditional branches, so the 24 keys are only a baseline. For example, with other conditions unchanged, x252 and x253 are added starting with SDK 28; optional collection results can add or remove further keys. The following discusses only the hash and encoding relationships that affect reproduction.

9.1 x86: XXH3-128 and CRC32

x86 = uppercase_hex(XXH3_128bits_withSeed(input, 0x48FA5412), 32)
      || uppercase_hex(CRC32(input), 8)

XXH3 uses the standard seeded 128-bit algorithm, with the result formatted as high64 << 64 | low64. CRC32 uses the standard reflected IEEE form and always outputs 8 hex characters.

The input is formed by concatenating successfully collected values in order, with no separators:

android_id ASCII
→ 非空 gcbooster_uuid、pps_oaid 的规范化字节
→ OAID ASCII（对应提供者分支启用时）
→ attestation 原始字节（采集分支存在时）
→ SoC serial（存在时）
→ resolution ASCII，例如 1080,1920,480
→ MMC cid（存在时）
→ UFS id（存在时）

SoC/MMC/UFS and the Settings values above are normalized by removing ASCII whitespace from both ends and then taking the first 200 bytes. cpuinfo and boot_id are not part of this input chain.

Fields of this kind are best confirmed by single-variable differentials: hold all other collection items constant, change one input, and observe the concatenation buffer and final hash. Evidence for a fixed seed should also include immediate-value loads in native, rather than relying only on fitting the outputs. The short-, medium-, and long-input branches of XXH3 should use a standard implementation; two 64-bit hashes should not simply be concatenated as a substitute for 128-bit output.

x86The combination of has only one line,xxh3_128The complete implementation of is given in Appendix B:

import zlib

def x86(data: bytes, seed: int = 0x48FA5412) -> str:
    return f'{xxh3_128(data, seed):032X}{zlib.crc32(data):08X}'

9.2 x137: MD5 of JNI method IDs

Encode the following seven JNI method IDs in order as little-endian u32 values, concatenate them into 28 bytes, compute standard MD5, and output uppercase hex:

Map.size()I
Map.get(Ljava/lang/Object;)Ljava/lang/Object;
Map.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
Map.remove(Ljava/lang/Object;)Ljava/lang/Object;
Map.keySet()Ljava/util/Set;
Map.values()Ljava/util/Collection;
Map.entrySet()Ljava/util/Set;

The hash formula and the source of the IDs must be considered separately.

The real-device path is established. Native calls java/util/Map with FindClass, then calls GetMethodID, using str x0 to write the 64-bit jmethodID into the global table. MD5 then reads only the first 4 bytes of each slot, namely the low 32 bits in little-endian order. All pointers in the device sample are below 4 GB, so their high 32 bits are zero and trunc32 equals the full pointer; this must not be generalized to every 64-bit address layout.

These IDs are addresses of ART method objects, located in an approximately 2.8 MiB, fileless rw- mapping. The seven values remain unchanged across process restarts during the same boot; after a device reboot, the mapping base is randomized and all seven values shift together while their relative offsets remain unchanged. Thus x137 is a runtime variable for the duration of a boot, and the server cannot strictly validate it against a fixed table. The simulation environment can still use Java hashCode of the descriptors as a substitute for the IDs, but only as a fallback oracle, not as ART's allocation formula.

x137:

import hashlib, struct

def x137(method_ids) -> str:
    return hashlib.md5(struct.pack('<7I', *method_ids)).hexdigest().upper()

9.3 attestation and environment branches

x147 decodes the attestation bytes as UTF-8, replacing invalid sequences with U+FFFD. x86 uses the raw bytes; it must not use x147 re-encoded after decoding.

def x147(attestation_bytes: bytes) -> str:
    return attestation_bytes.decode('utf-8', errors='replace')

The input source is established. Native uses Class.forName to load com.huawei.attestation.HwAttestationManager, then reflectively invokes getDeviceID(I)[B. This class is not in the APK; it is a Huawei system class. When the class exists, x148 is an empty string and the raw bytes enter x86; if class loading fails, neither x147 nor x148 appears, and the attestation bytes do not enter x86 either. After the failure, it also probes attestation_service and IHwTelephony$Stub, but they cannot substitute for the return value of getDeviceID.

On the Xiaomi MI 6 (Lineage, with no Huawei package), all three process restarts took the omission branch: the x147 path in the registration snapshot produced no bytes. In the same process, AMediaDrm_getPropertyByteArray("deviceUniqueId") consistently returned 32 bytes, but that is a Widevine collection; when HwAttestationManager was hidden, x147 still disappeared, so it is not an input to x147. The Huawei TEE's getDeviceID payload remains device-specific and uncovered.

The simulation environment still defaults to SHA256(UTF8(deviceId)) as the fallback; raw bytes may also be injected, or the omission branch selected explicitly.

Likewise, the OAID provider, file readability, and JNI environment can change the snapshot. Reproducing serialization and encryption for one fixed environment does not mean that the collection behavior of every device manufacturer has been reproduced.

9.4 x165: String encoding of filesystem information

x165 encodes the file status and filesystem status of 12 fixed targets into an object. Successful items use one-based decimal index strings as keys,/sdcard corresponding to "12". The format of each item is:

{ctime_sec}{ctime_nsec:09d}-{inode}-{device}-{fsid}-{fstype}

The fixed indices and targets are as follows; these are Android collection targets, not directories on the reader's computer:

On this MI 6, the 12 targets include /data/system, /data/data/, /data/data/com.android.shell, /data/system/install_sessions, /sdcard, /dev/fd/0, /dev/fd/1 and the two radio/debuggerd property nodes are present; webview, the GMS data directory, and ffs_prop. /system/bin/getprop and /system/bin/sh are both present; under the helper rule in Section 9.5, this device's x128 should take the omission-key branch. This has not yet been checked directly against a registration snapshot taken after clearing the data.

There is no separator between the seconds and nanoseconds of ctime; nanoseconds are at least 9 digits and are zero-padded as needed. Both are output as signed 64-bit integers. inode, device, and fstype are output as unsigned 64-bit integers. fsid encodes each of two u32 values as 8 lowercase hex characters and concatenates them in low, high order.

When stat fails but statfs succeeds, ctime, inode, and device are set to zero, while fsid and fstype are retained; when stat succeeds but statfs fails, fsid is the literal *, and fstype is 0. If both fail, the item is omitted. If all targets fail, x165 is null rather than an empty object.

For example, if only item 12 is retained, all stat values are zero, and the two fsid words are 0xd3609fe8, 0x04970d6b, while fstype is the decimal value 61267, the result is:

{"12":"0000000000-0-0-d3609fe804970d6b-61267"}

9.5 x170 and x128 conditional branches

In the observed class-loading experiments, when com.android.id.impl.IdProviderImpl is present, x170 is "id_provider", OAID is also written to x173, x174, x175, and x181, and appended once in x86. After the class is hidden, x170 falls back to "vivo", the four copies disappear, and x86 no longer appends OAID; x171 still retains OAID. This is the branch result in the simulated environment and cannot be used to infer the provider on a real device from the manufacturer string alone.

x128 corresponds to the errno=2 text after a file-open failure, "No such file or directory". In the observed experiments, this key disappeared when both getprop and sh system helpers were available; it remained when only getprop was available. boot_id follows a separate path through x47; in the samples, its trailing newline was removed and the first 35 characters were retained, and it does not participate in x86. The native code's complete handling of abnormal lengths and encodings has not yet been covered.

9.6 Snapshot fields and presence conditions

The following fields have been mapped. Strings are serialized as UTF-8, and capacity fields are byte-count integers; “present” means that the collection branch succeeded or that the caller explicitly supplied the input.

SoC serial, MMC cid, and UFS id correspond respectively to /sys/devices/soc0/serial_number, /sys/block/mmcblk0/device/cid, and /sys/ufs/ufsid; their normalized bytes are included in x86, while the field values decode those bytes as UTF-8 with replacement. The two representations cannot be interchanged for invalid UTF-8 input. When publicSourceDir is not provided separately, the current reconstruction strategy uses sourceDir.

Settings name

10. Combining the layers into a single request

After session bootstrapping is complete, a request is processed in the following order:

Determine the final method, path, query, and body.

1. Determine the final method, path, query, and body.
2. Take the counter for this request and generate MUA from the session's s and scalar together with the current profile.
3. For a registration request, use the same session's slotA to generate d and the final body before proceeding.
4. Assemble canonical and compute SIG and S1 separately; S1 uses the same counter and the current Unix-second timestamp.
5. Add session identity, common params, scene, and other request headers.
6. Compute Shield from the final path, query, allowlisted headers, and body.
7. Send the request using the exact bytes involved in signing.

There are three kinds of state here that are easy to confuse: the session randomness used by MUA, the counter for each request, and Shield's nonce4. They have different lifecycles and cannot substitute for one another.

The same request object should provide the bytes needed for both canonical and the Shield source. If the HTTP library reorders the query, changes percent-encoding, or reserializes the body after signing, the result will be that the local formula is correct but the content actually sent is different. Registration in particular requires generating the final body before computing SIG and S1.

10.1 A Reproducible Set of SIG / S1 Inputs

The following is a synthetic vector for checking the formulas. MUA uses the literal part1.part2., to isolate SIG/S1; it does not represent a real MUA that can be submitted to the server.

method    = GET
path      = /api/sns/v1/note/imagefeed
query     = note_id=example
body      = 空字节
mua       = part1.part2.
counter   = 2
timestamp = 1700000000

Construct canonical according to the LF concatenation rule described in this article. The result should be:

SHA256(canonical) =
06fc306a81f499526239ed443e542e9a0f25df4625d7ca5a26020e8219cd697a

SIG =
7a78e9c4e104f6a7ffcc6e504d5fe6720f25df4625d7ca5a26020e8219cd697a

S1 lane32 =
390c3d289aa264822327b6ef0b47391162ebd4d38caa8da24eaa90dd4e9a9462

S1 raw62 =
000200000001a2b04bc749cdebd7dea41d703b71619b9b3b088e5cff71d06ee72b986dfd5736a04399afdf3836d7a3937736b3df1fad51852ee062ebcfea

S1 Base64 =
AAIAAAABorBLx0nN69fepB1wO3Fhm5s7CI5c/3HQbucrmG39VzagQ5mv3zg216OTdzaz3x+tUYUu4GLrz+o=

This vector was cross-checked against two implementations and is intended to verify bit order, counter, time units, and encoding. A native full-frame comparison with counter=2 at a fixed time was performed separately; see Section 4.4.

The existing fixed-input comparisons support the MUA encoding, SIG, S1, main_hmac decryption, and Shield formulas described here. Beyond formula-level comparisons, a pure-Python client completed end-to-end validation in an anonymous session: it obtained a GID during registration, recovered main_hmac and main_ssk, and received code=0 from homefeed, imagefeed, search/notes, user/info, and user/posted across several synthetic devices and sessions. This validates that the server accepts the constructed headers and the associated state transitions. The real-device input for x137 has been identified as a boot-lifetime ART address (see Section 9.2); areas not yet covered include the real TEE attestation bytes and the complete OEM collection branches. Those gaps limit the scope of the conclusions and cannot be filled by a single successful server response.

Appendix: SIG Affine Matrix

Each row contains 129 bits placed in the upper 129 bits of 17 bytes in the order “column 0 through column 128”; the lower 7 bits are zero-filled. The hex below totals 2176 bytes and 128 rows; for formatting, each text line contains four matrix rows.

0000006b000000a1000000f41aa7488200000000d600000043000000e9344f900480000000ad00000087000000d3699e200a000000005a0000000f000000a7d23d401480
000000de000000be000000bbbfddc9a880000000d7000000dc00000083651ddbd000000000af000000b800000007cb3ab6a08000000035000000d0000000fa8dd324c080
000052c2000050580000948e724ad6de000000a4840000a0b10000281ce494adbc00000049090000416200005138c8285a788000009213000083c50000a2709151b5f280
000076e5000057d20000d16e50e8bc38800000be080000fefd00003752d29bafac0000007d110000fcfa00006fa4a5375e58000000a9e10000a8ac00004ac739256b6e80
005903f600b6dfcb67928b7da4d5840e0000b306ed006cbe97cf2416fb49ab091c8000660cdb00d87d2e9f482df69256133a0000cd19b600b0fb5d3f905bed24ac277400
00c2319a00d6297019b23ca6ec8dcae60000dd60c3001b8c2b55f6f3307dcf11c20000bac18600361857aaede761fb9f238400002c81fb00dbef65334945be52eac20680
2e83b6e02070089b2805fb342ddca9ea805d076cc040e01137510af6695bb852d680bb0ed88081c0236fa215edd2b771a4ae80771db001038046de452bdba56ee2495e00
c0b9d6e226718527a3524d7ff1183b5600aef01b246c9203d56fa161cbceecdf46805ce13648d82406abde43c3979dd9be8e809741db70903804cd94827d1a166ed4f480
0000c839000004bf525057d5373bf6f200000091720000087fa4a0aeab6f76ede480000022e4000011ff49415c56deecdbca00000044c8000023fe9283b9acbcd9b69480
000041a8000042437657248d4e899ad80000004b6900008138befe1fcfab29c34200000096d2000003707dfc3e9f56538684800000e49c0000025fa9a82bea9b9dfbf880
0000e80700007f41c7a6066322a26ad2000000d10e0000ff838e4d0cc74445d4a4800000a31d0000fe071c9a198e898ba94a800000473a0000fc0e3835321c1216529600
000067730000875cb7cd635b078fcefe00000027e1000070f8a93dc1d42cbcf72e0000004ec20000e0f0527a82a95979ef5c80000074830000bfa0635303319151b56880
0061cc240066d036e82b4c58155412588000c3994800cda16dd15798b12ba825b28000863390009b43daa3ae316257504a6600000d6720003787b4475c63c5aea094cc80
007a03640009df5e67928bd249153ac2000094caed00756e8b270f5afd877f66dc80002895db00eadc164e1fb5fa0fffcdba000030e69200b3681b741526ac0aaa892c00
4f7dffe1cd22b109585c8fcce37ac080809efbfec29b446213b1b91e99c6f58002803df6fc843789c52762733c338ceb0106007aedf8096f128a4ec5e6786718d7020c00
baa60ff21307a595d2907e03d3d5c498003a30e004ea2cfa23fd7c72ca44d148b0007461c008d559f446faf9e49589a3906000a7be7ff066915884acae47e6f13de04000
004972c159e314b3a0d85844c737f8de000092e482b3c6296641b0b1898e6ff0bc000024c805668c53cd836062121cdee178800048910acd18a69b07c1c52538bcc2f280
00d950d4c2d35984ae5ad20eb74e7c388000fbd268dd44a7bafc6dfd59a9ab01ac0000f6a5d0ba894f75f8dafab3525602580000a439602cf18a59506cac22639bfc6e80
00910f4e61cbf6b4a99b5dc62bf114300000221f9cc397ed685237bb8c57e329608000443e39862edbd0a46f7718aec653c28000897c720d5db6a149deee305c8ca68600
0083f6aa7a709af73b2780a792e9593c800096e21a942bc35bdfd55c890f22a74a00002cc434285786b7beabb9121f444f940000c887273065fbdad4cd2ee315788a1800
0026917e7f7a7f7fcc52e223de0bd9a880004c22fdfff5ffff99a4c446bc17b25200009844fafeebfefe3349888d792f64a400003189f4fcd7fcfc6792101af35fc94880
0045839787d587870376c31739b44a3a8000ad965070d17070cabe640cac634dde80005a2ca0e0a3e0e0957dc91958c79abe000093c83fbf3dbfbfe6a971116f85ecd400
743ecdc06bec892c6fddd951127d57b600e87c9b80d6d91259debab2a225fbae6c00d1f93701adb225b3bc7564454af65cd800a3f26f025a644a6679eac98b94edb9b000
33db13c4de251de19c084a473aa624d6001388ea48d7a6b3ee56cc4dde66301f1a802710d590af4d66ddac999abccd613e36003a1f66e03576449637eeec2889be2bda80
c751e0217a1dd8365c17560d304c03f4008ea2c042f53ab06db92fac1b619806e8801c458085eb7460da735f5836c3310cd280388b010bd7e8c1b4e6bfb16d866319a680
b747e236d5cc5a5e906834d73d8b31ba80a9de244cd1846d8b7cc73fa24a5a60828052bc4898a309da16f98e7e4594b5c10680632870103d0e6c1bae0bab86182681fa00
2ec92c906de01e859dd433f9fee74048005d935920dac03c0b3ba967f2fcce819000bb26b340b48078177652cfe5f89d032000774c66816801f12feca49fcaf03b064000
c051e193bde2fdda449d0c6c1f914cc880ae6beeb61624e43114ef2a21c0c5d8da805cd6dd6c2d48c86329df5542808ab0b6009764964836708f42ce6a997cfff3202400
5c68805b2d88b57c09115555c24ba8c400b9d001b75b106af91323aaaa849650880073a1026eb721d4f227465454092ca01000e64304dc6e42a9e54e8da8a81359412080
90ef88e3f10de7b6950b0505e5f82b86007cb7909dce937b1023065f5f08bbffc880f96e203b9d26f721460cbfbf1177fe9280aeb4c02d16c45a3e84082a2ae1a554e200
3cf10ea808ea48249e0228807d47ed4e0078e31d5011d590483d045101fb8fdb9c00f1c63aa023ab20907a08a202f61eb63880e38c744146564020f5114504ed3c6c7200
fae9e62b8546c9647521a388a63f35aa00c822c2ff0367dbed74406f903039861a00914484fe06cfb6dbe881de2061720d34001e780754047524924f0194c0bea3f62680

Decoding method:

def unpack_matrix(hex_text):
    raw = bytes.fromhex(hex_text)
    if len(raw) != 128 * 17:
        raise ValueError("expected 2176 bytes")
    rows = []
    for offset in range(0, len(raw), 17):
        value = int.from_bytes(raw[offset:offset + 17], "big") >> 7
        rows.append([(value >> bit) & 1 for bit in range(128, -1, -1)])
    return rows

This bit-packing order is used only to store the matrix. After decoding to rows, calculate using the input-column bit order from section 3.2; do not mistake the most-significant-bit-first packing order for the order in which bits are taken from the digest.

Appendix B: Complete XXH3-128 Branches

On x86, this uses the seeded XXH3-128 from xxHash 0.8.3. What is expanded here is the standard hash implementation; what is specific to this protocol is the seed 0x48FA5412, the input concatenation order, and the rule for appending CRC32 at the end.

The implementation normalizes the seed to an unsigned 64-bit value. Integers are read in little-endian order, and 64×64 multiplication retains the full 128-bit result; when folding is required, the high and low 64-bit halves of the product are XORed. Truncation after addition, multiplication, and negation follows the MASK in the code.

The medium-length mix16 uses (secret_low + seed) and (secret_high - seed) to mask the two input u64 values, then folds the product into 64 bits. mix32 updates low/high simultaneously and cross-XORs the sum of the other half's two words. The final window in the 129–240-byte branch uses the negated seed; its position and direction cannot be swapped with those of the preceding full group.

For long inputs with a nonzero seed, the implementation first derives a 192-byte secret: the first u64 of each 16-byte unit is increased by the seed, and the second u64 is decreased by the seed. Each stripe is split into eight lanes; the original values are added to adjacent accumulators, while the two u32 halves of the masked values are multiplied and added to the current accumulator. After each complete 1024-byte block, the state is scrambled. The final 64-byte window is processed separately and may overlap the preceding window. In the end, high and low are merged using different secret offsets, so this is not two independent 64-bit hashes.

The code below runs independently using only Python's built-in features. It is a Python adaptation derived from xxHash 0.8.3; the original algorithm is copyright Yann Collet and remains available under the BSD-2-Clause license. The complete license text appears at the end.

MASK = (1 << 64) - 1
P1 = 0x9E3779B185EBCA87
P2 = 0xC2B2AE3D27D4EB4F
P4 = 0x85EBCA77C2B2AE63
SECRET = bytes.fromhex("b8fe6c3923a44bbe7c01812cf721ad1cded46de9839097db7240a4a4b7b3671fcb79e64eccc0e578825ad07dccff7221b8084674f743248ee03590e6813a264c3c2852bb91c300cb88d0658b1b532ea371644897a20df94e3819ef46a9deacd8a8fa763fe39c343ff9dcbbc7c70b4f1d8a51e04bcdb45931c89f7ec9d9787364eac5ac8334d3ebc3c581a0fffa1363eb170ddd51b7f0da49d316552629d4689e2b16be587d47a1fc8ff8b8d17ad031ce45cb3a8f95160428afd7fbcabb4b407e")


def _read(b: bytes, pos: int) -> int:
    return int.from_bytes(b[pos:pos + 8], "little")


def _avalanche(x: int) -> int:
    x &= MASK
    x ^= x >> 37
    x = x * 0x165667919E3779F9 & MASK
    return x ^ (x >> 32)


def xxh3_128_medium(data: bytes, seed: int = 0) -> int:
    """Return high64<<64|low64, rejecting lengths outside 17..240."""
    n = len(data)
    if not 17 <= n <= 240:
        raise ValueError("medium XXH3-128 currently supports 17..240 bytes")
    seed &= MASK
    low, high = n * P1 & MASK, 0

    def mix16(pos: int, sec: int, seed: int) -> int:
        a = _read(data, pos) ^ ((_read(SECRET, sec) + seed) & MASK)
        b = _read(data, pos + 8) ^ ((_read(SECRET, sec + 8) - seed) & MASK)
        product = a * b
        return ((product & MASK) ^ (product >> 64))

    def mix32(a: int, b: int, sec: int, sd: int) -> None:
        nonlocal low, high
        low = ((low + mix16(a, sec, sd)) & MASK) ^ ((_read(data, b) + _read(data, b + 8)) & MASK)
        high = ((high + mix16(b, sec + 16, sd)) & MASK) ^ ((_read(data, a) + _read(data, a + 8)) & MASK)

    if n <= 128:
        for i in range((n - 1) // 32, -1, -1):
            mix32(16 * i, n - 16 * (i + 1), 32 * i, seed)
    else:
        for i in range(32, 160, 32):
            mix32(i - 32, i - 16, i - 32, seed)
        low, high = _avalanche(low), _avalanche(high)
        for i in range(160, n + 1, 32):
            mix32(i - 32, i - 16, 3 + i - 160, seed)
        mix32(n - 16, n - 32, 136 - 17 - 16, -seed & MASK)
    result_low = _avalanche(low + high)
    result_high = -_avalanche(low * P1 + high * P4 + (n - seed) * P2) & MASK
    return result_high << 64 | result_low


P3 = 0x165667B19E3779F9
P5 = 0x27D4EB2F165667C5
P32_1 = 0x9E3779B1
P32_2 = 0x85EBCA77
P32_3 = 0xC2B2AE3D


def _swap(x: int, size: int) -> int:
    return int.from_bytes(x.to_bytes(size, "little"), "big")


def _avalanche64(x: int) -> int:
    x &= MASK
    x = ((x ^ (x >> 33)) * P2) & MASK
    x = ((x ^ (x >> 29)) * P3) & MASK
    return x ^ (x >> 32)


def _short(data: bytes, seed: int) -> int:
    n = len(data)
    if n == 0:
        low = _avalanche64(seed ^ _read(SECRET, 64) ^ _read(SECRET, 72))
        high = _avalanche64(seed ^ _read(SECRET, 80) ^ _read(SECRET, 88))
    elif n < 4:
        combined = (data[0] << 16) | (data[n >> 1] << 24) | data[-1] | (n << 8)
        swapped = _swap(combined, 4)
        rotated = ((swapped << 13) | (swapped >> 19)) & 0xFFFFFFFF
        flip_l = (int.from_bytes(SECRET[:4], "little") ^ int.from_bytes(SECRET[4:8], "little")) + seed
        flip_h = (int.from_bytes(SECRET[8:12], "little") ^ int.from_bytes(SECRET[12:16], "little")) - seed
        low = _avalanche64(combined ^ (flip_l & MASK))
        high = _avalanche64(rotated ^ (flip_h & MASK))
    elif n <= 8:
        seed ^= _swap(seed & 0xFFFFFFFF, 4) << 32
        value = int.from_bytes(data[:4], "little") | (int.from_bytes(data[-4:], "little") << 32)
        flip = ((_read(SECRET, 16) ^ _read(SECRET, 24)) + seed) & MASK
        product = (value ^ flip) * (P1 + (n << 2))
        low, high = product & MASK, product >> 64
        high = (high + (low << 1)) & MASK
        low ^= high >> 3
        low = ((low ^ (low >> 35)) * 0x9FB21C651E98DF25) & MASK
        low ^= low >> 28
        high = _avalanche(high)
    else:
        flip_l = ((_read(SECRET, 32) ^ _read(SECRET, 40)) - seed) & MASK
        flip_h = ((_read(SECRET, 48) ^ _read(SECRET, 56)) + seed) & MASK
        head, tail = _read(data, 0), _read(data, n - 8)
        product = (head ^ tail ^ flip_l) * P1
        low, high = product & MASK, product >> 64
        low = (low + ((n - 1) << 54)) & MASK
        tail ^= flip_h
        high = (high + tail + (tail & 0xFFFFFFFF) * (P32_2 - 1)) & MASK
        low ^= _swap(high, 8)
        product = low * P2
        low = _avalanche(product & MASK)
        high = _avalanche((product >> 64) + high * P2)
    return (high << 64) | low


def _long(data: bytes, seed: int) -> int:
    if seed:
        parts = []
        for i in range(0, 192, 16):
            parts.append(((_read(SECRET, i) + seed) & MASK).to_bytes(8, "little"))
            parts.append(((_read(SECRET, i + 8) - seed) & MASK).to_bytes(8, "little"))
        secret = b"".join(parts)
    else:
        secret = SECRET
    acc = [P32_3, P1, P2, P3, P4, P32_2, P5, P32_1]

    def stripe(pos: int, sec: int) -> None:
        for lane in range(8):
            value = _read(data, pos + 8 * lane)
            keyed = value ^ _read(secret, sec + 8 * lane)
            acc[lane ^ 1] = (acc[lane ^ 1] + value) & MASK
            acc[lane] = (acc[lane] + (keyed & 0xFFFFFFFF) * (keyed >> 32)) & MASK

    n = len(data)
    blocks = (n - 1) // 1024
    for block in range(blocks):
        for j in range(16):
            stripe(block * 1024 + j * 64, j * 8)
        for lane in range(8):
            value = acc[lane]
            acc[lane] = ((value ^ (value >> 47) ^ _read(secret, 128 + lane * 8)) * P32_1) & MASK
    for j in range(((n - 1) - blocks * 1024) // 64):
        stripe(blocks * 1024 + j * 64, j * 8)
    stripe(n - 64, 121)

    def merge(sec: int, start: int) -> int:
        total = start
        for i in range(4):
            product = (acc[2*i] ^ _read(secret, sec+16*i)) * (acc[2*i+1] ^ _read(secret, sec+16*i+8))
            total += (product & MASK) ^ (product >> 64)
        return _avalanche(total)

    return merge(117, ~(n * P2)) << 64 | merge(11, n * P1)


def xxh3_128(data: bytes, seed: int = 0) -> int:
    """Return the unsigned 128-bit digest (high64 followed by low64).

    Seed is reduced modulo 2**64, as in the xxHash C API. This is a one-shot
    implementation, with no runtime dependency on xxhash/unidbg/native code.
    """
    if not isinstance(data, (bytes, bytearray, memoryview)):
        raise TypeError("data must be bytes-like")
    if type(seed) is not int:
        raise TypeError("seed must be an integer")
    data = bytes(data)
    seed &= MASK
    if len(data) <= 16:
        return _short(data, seed)
    if len(data) <= 240:
        return xxh3_128_medium(data, seed)
    return _long(data, seed)

The return value is high64 << 64 | low64. When generating x86, format this integer as fixed-width 32-bit uppercase hex, then append the fixed-width 8-character uppercase CRC32 of the same input:

import zlib

def make_x86(data):
    return f"{xxh3_128(data, 0x48FA5412):032X}{zlib.crc32(data):08X}"

License:

Copyright (C) 2012-2023 Yann Collet

 BSD 2-Clause License (https://www.opensource.org/licenses/bsd-license.php)

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are
 met:

    Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    Redistributions in binary form must reproduce the above
      copyright notice, this list of conditions and the following disclaimer
      in the documentation and/or other materials provided with the
      distribution.

 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 You can contact the author at:
   - xxHash homepage: https://www.xxhash.com
   - xxHash source repository: https://github.com/Cyan4973/xxHash