Summary
microsoft/Olive#2584 (extending RTN quantization to MoE experts via QuantTensor) went through 3 rounds of adversarial review. Round 3 found 5 new Major-severity bugs, all independently reproduced, but not fixed in that PR — tracked here as a separate follow-up so #2584 can merge without blocking on them.
Why these are follow-ups, not blockers for #2584
All 5 are edge cases in QuantTensor's live in-memory dispatch (tensor.py) or the skip-pattern regex safety check (patterns.py) — neither is exercised by the currently-supported RTN quantization pipeline, which is @torch.no_grad() round-to-nearest and never runs a live forward pass through QuantTensor. They matter for: (a) future passes that do run forward calibration through quantized MoE weights (e.g. GPTQ, see microsoft/Olive#<gptq-issue>), and (b) any direct PyTorch-side use of QuantTensor outside the quantize-then-save flow.
The 5 issues
tensor.py — uint8-dtype tensor indices misclassified as safe integer indices. Same bug class as a round-2 fix for boolean-mask indexing, different dtype variant not covered by that fix.tensor.py — copy_() does not propagate/clear is_placeholder. Copying real data into a placeholder QuantTensor can still get silently no-op'd by a later init-style op (zero_, normal_, ...).state_dict.py — refresh_quant_tensor_refs clears is_placeholder unconditionally, not gated on an actual data load having completed.tensor.py — arbitrary-rank integer indexing (added to support top-k MoE routing) can produce a QuantTensor that can't be dequantized or re-indexed afterward.patterns.py — a third ReDoS bypass via (?#...) inline-comment regex syntax, confusing the char-class scanner used by the skip-pattern safety check (previous two bypasses: (a{1,2})+$, ((a|aa))+$).
Structural observation
This is the third consecutive round where a whitelist/blacklist-enumeration defense (index-dtype whitelist in tensor.py, regex-construct blacklist in patterns.py) gets bypassed by a new variant not covered by the previous fix. This is a signal the enumeration approach may be the wrong shape of fix for both checks, not that a 4th patch will finish the job. A more robust alternative — post-hoc shape/data assertion (verify the actual resulting shape/rank against what was claimed, after the op runs, rather than enumerating safe forms upfront) — was proposed but not implemented; there's a partial existing precedent for this pattern at olive/passes/onnx/compose.py:173.
References
Summary
microsoft/Olive#2584(extending RTN quantization to MoE experts viaQuantTensor) went through 3 rounds of adversarial review. Round 3 found 5 new Major-severity bugs, all independently reproduced, but not fixed in that PR — tracked here as a separate follow-up so #2584 can merge without blocking on them.Why these are follow-ups, not blockers for #2584
All 5 are edge cases in
QuantTensor's live in-memory dispatch (tensor.py) or the skip-pattern regex safety check (patterns.py) — neither is exercised by the currently-supported RTN quantization pipeline, which is@torch.no_grad()round-to-nearest and never runs a live forward pass throughQuantTensor. They matter for: (a) future passes that do run forward calibration through quantized MoE weights (e.g. GPTQ, seemicrosoft/Olive#<gptq-issue>), and (b) any direct PyTorch-side use ofQuantTensoroutside the quantize-then-save flow.The 5 issues
tensor.py— uint8-dtype tensor indices misclassified as safe integer indices. Same bug class as a round-2 fix for boolean-mask indexing, different dtype variant not covered by that fix.tensor.py—copy_()does not propagate/clearis_placeholder. Copying real data into a placeholderQuantTensorcan still get silently no-op'd by a later init-style op (zero_,normal_, ...).state_dict.py—refresh_quant_tensor_refsclearsis_placeholderunconditionally, not gated on an actual data load having completed.tensor.py— arbitrary-rank integer indexing (added to support top-k MoE routing) can produce aQuantTensorthat can't be dequantized or re-indexed afterward.patterns.py— a third ReDoS bypass via(?#...)inline-comment regex syntax, confusing the char-class scanner used by the skip-pattern safety check (previous two bypasses:(a{1,2})+$,((a|aa))+$).Structural observation
This is the third consecutive round where a whitelist/blacklist-enumeration defense (index-dtype whitelist in
tensor.py, regex-construct blacklist inpatterns.py) gets bypassed by a new variant not covered by the previous fix. This is a signal the enumeration approach may be the wrong shape of fix for both checks, not that a 4th patch will finish the job. A more robust alternative — post-hoc shape/data assertion (verify the actual resulting shape/rank against what was claimed, after the op runs, rather than enumerating safe forms upfront) — was proposed but not implemented; there's a partial existing precedent for this pattern atolive/passes/onnx/compose.py:173.References
jambayk/moe-quant: extend RTN weight quantization to MoE experts #2583