Skip to content

[Bug]: MXFP LLMC format is not correct. #1629

@xin3he

Description

@xin3he

Problem Description

AutoRound MXFP config is not correct. so I used the config that Yi generated. here.https://huggingface.co/INC4AI/Qwen3-8B-MXFP8-LLMC-TEST-ONLY/blob/main/config.json#L59

For example, the scale dtype in AR is torch.float8_e4m3fn

Reproduction Steps

pytest test_cuda/export/test_llmc_format.py::TestAutoRound::test_mxfp8_llmcompressor_format with branch xinhe/3-27b

Environment Information

No response

Error Logs

"quantization_config": {
    "config_groups": {
      "group_0": {
        "format": null,
        "input_activations": {
          "actorder": null,
          "block_structure": null,
          "dynamic": "local",
          "group_size": 32,
          "is_mx": true,
          "num_bits": 4,
          "observer": "static_minmax",
          "observer_kwargs": {},
          "scale_dtype": "torch.float8_e4m3fn",
          "strategy": "tensor_group",
          "symmetric": true,
          "type": "float",
          "zp_dtype": null
        },
        "output_activations": null,
        "targets": [
          "Linear"
        ],
        "weights": {
          "actorder": null,
          "block_structure": null,
          "dynamic": false,
          "group_size": 32,
          "is_mx": true,
          "num_bits": 4,
          "observer": "memoryless_minmax",
          "observer_kwargs": {},
          "scale_dtype": "torch.float8_e4m3fn",
          "strategy": "tensor_group",
          "symmetric": true,
          "type": "float",
          "zp_dtype": null
        }
      }
    },
    "format": "float-quantized",
    "global_compression_ratio": null,
    "ignore": [],
    "kv_cache_scheme": null,
    "provider": "auto-round",
    "quant_method": "compressed-tensors",
    "quantization_status": "compressed"
  },

Additional Context

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions