History of Stability AI
Founded in 2019 and based in London, Stability AI is a pioneering open-source AI company. They are dedicated to creating cutting-edge technologies that benefit both humanity and the world. In 2022, Stability AI released Stable Diffusion, an image model that represents a transparent open and scalable alternative to proprietary AI. They are continuing to make foundational AI technology accessible to all through their work on Stable LM.
Stable LM Alpha Release
Stability AI has released their own large language model called Stable LM. The models were trained on a data set called "the pile," which includes data from Wikipedia, YouTube, and Reddit. It is completely free and open source, including commercially viable. The model comes in a 3 billion parameter and a 7 billion parameter version, with larger models coming soon. Developers can freely inspect, use, and adapt the base models for commercial or research purposes subject to the terms of the CC BY-SA-4.0 license. The current release is an alpha version, but they plan to add RLHF (Reinforcement Learning with Human Feedback) technology used by OpenAI in their next iteration to improve performance.
Features of Stable LM
Stable LM models can generate text and code and will power a range of downstream applications. The models are trained on an experimental dataset built on the pile which is three times larger than previous datasets used for training language models. Despite its small size (3 to 7 billion parameters), stable LM has surprisingly high performance in conversational encoding tasks. For comparison, GPT-3 is a 175 billion parameter model. Despite being much smaller than GPT-3, Stability AI's seven billion parameter model performs better than GPT-3 in some tasks. Stability AI plans to release more powerful models ranging from 16 billion to 65 billion parameters.
Possible problems
As per document on GitHub, it claims that like any other large language model that hasn’t been fine-tuned or reinforced, the responses you receive may vary in quality and may even contain offensive language or views. However, with more data, community feedback, and optimization, the quality of responses is expected to improve.
source - https://github.com/stability-AI/stableLM/#potential-issues
Interacting with Models
There are two ways to interact with the models through Hugging Face Hub and by running a provided notebook on your own machine. A notebook is available with code and instructions on how to get started. The T4 GPU used has 15 gigs of RAM. While running on Google Colab, it may require a pro version due to high system requirements. When running on a local machine, high capacity or high RAM machine may be necessary. All necessary links are provided under 'source' section at end.
The Hugging Face Hub provides access to a demo of the fine-tuned version of the seven billion parameter model.
Testing Stable LM Base Model Alpha 7B
Take a look at the examples below, which were generated by Stability AI's fine-tuned model with 7 billion parameters.
The launch of Stable LM Alpha by Stability AI is a monumental milestone in the domain of language models. This remarkable creation boasts a captivating background, exceptional features, and user-friendly ways to interact with the models. With such innovations, the use of AI is bound to undergo a transformative shift. Although there exist certain predicaments that demand resolution, the potential for this cutting-edge technology is immensely encouraging.
No comments:
Post a Comment