Loading mpt-7b-storywriter-4bit-128g Model Type
If you are trying to load the mpt-7b-storywriter-4bit-128g model and encounter an error message saying it can't determine the model type, don't worry, you're not alone. Several users have experienced the same issue, and the good news is there are ways to resolve it.
One of the main reasons why you might get the error message is that the model type is "mpt," which was created by the developers themselves and is not one of the model types with GPTQ support.
To resolve the issue, you can specify the model type manually using the --model_type argument. This will help the system to identify the model type correctly.
Another solution is to try loading the original mosaicml/mpt-7b-storywriter model instead of the OccamRazor/mpt-7b-storywriter-4bit-128g model. Several users have reported success with this method.
If you're still having issues, it's a good idea to check out the comments and discussions on similar issues such as #1856 and #1815. You might find a solution that works for you.
By following these tips, you should be able to load the mpt-7b-storywriter-4bit-128g model type without any further issues. Happy modeling!
