Search results

Compatibility with transformers > 4.36: error: AttributeError: 'tuple' object has no attribute 'to_legacy_cache'

Status: Open.

#137 In BlackSamorez/tensor_parallel;

· Dr-Left opened on Aug 21, 2024

Customized generate func support?

Status: Open.

#136 In BlackSamorez/tensor_parallel;

· MonolithFoundation opened on May 1, 2024

tensor_parallel int4 LLM is not working since release v2.0.0

Status: Open.

#133 In BlackSamorez/tensor_parallel;

· ReinForce-II opened on Jan 2, 2024

Now, does tensor_parallel no longer support the huggingface trainer?

Status: Open.

#132 In BlackSamorez/tensor_parallel;

· HanGyeol-Yoo opened on Dec 31, 2023

Can I use tensor_parallel to inference for a GPTQ quantized model?

Status: Open.

#131 In BlackSamorez/tensor_parallel;

· minlik opened on Nov 15, 2023

No implement of generate() when using models from hugging face.

Status: Open.

#130 In BlackSamorez/tensor_parallel;

· 342215448 opened on Nov 14, 2023

TensorParallel object has no attribute save_pretrained

Status: Open.

#129 In BlackSamorez/tensor_parallel;

· toufunao opened on Nov 3, 2023

No output when using tensor_parallel

Status: Open.

#128 In BlackSamorez/tensor_parallel;

· yyya9 opened on Oct 27, 2023

AttributeError: object has no attribute 'devices'

Status: Open.

#125 In BlackSamorez/tensor_parallel;

· QiueY514 opened on Oct 9, 2023

ValueError: Model parameters were moved to incorrect devices, did call on model.cuda() or model.to(device)? If so, please avoid doing that

Status: Open.

#124 In BlackSamorez/tensor_parallel;

· Khyat opened on Oct 4, 2023

Max Recursion Error when using with lora

Status: Open.

#122 In BlackSamorez/tensor_parallel;

· Ar-Kareem opened on Oct 2, 2023

RuntimeError: NCCL Error 3: internal error

Status: Open.

#121 In BlackSamorez/tensor_parallel;

· smallmocha opened on Sep 12, 2023