How to install and Run GPT4ALL using Python in Local, No GPU Required !

Prerequisites: This blog is for curious minds looking to tap into the potential of Large Language Models (LLMs) to enhance both their daily lives and business Objective. All you need is a basic understanding of Python and a pinch of curiosity – no deep learning wizardry required!

Gpt4All is a free-to-use, locally running, privacy-aware chatbot. No GPU or internet required. GPT4ALL Official page : https://gpt4all.io/index.html. We will be using Gpt4all’s Python Bindings : https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md

Install Required Packages

pip install gpt4all==1.0.6

pip show gpt4all

pip install nomic

Please ensure you install this specific version as other versions might not support certain LLM models. Our focus here is on running models locally on the CPU, so we’ll be working with models in different formats, such as GGML and GGUF. You can download these models from the Hugging Face website here.

GGML models are a type of LLM that are designed to be more efficient and faster to run on CPUs. They do this by using a number of techniques

Download a GGML based Large Language Model from Hugging Face : http://huggingface.co/models

from gpt4all import GPT4All

model = GPT4All("/content/drive/MyDrive/My_System_LLM/text-generation-webui/models/llama-2-7b-chat.ggmlv3.q2_K.bin",)

After Model being Initialised, lets Try and give it a shot.

Lets give a Long text article from Wikipedia, it will take some due to such long tokens :

prompt = "Summarize the given text \n\n Text:%s"
text = """Ancient India

Manuscript illustration, c. 1650, of the Sanskrit epic Ramayana, composed in story-telling fashion c. 400 BCE – c. 300 CE[82]
By 55,000 years ago, the first modern humans, or Homo sapiens, had arrived on the Indian subcontinent from Africa, where they had earlier evolved.[27][28][29] The earliest known modern human remains in South Asia date to about 30,000 years ago.[27] After 6500 BCE, evidence for domestication of food crops and animals, construction of permanent structures, and storage of agricultural surplus appeared in Mehrgarh and other sites in Balochistan, Pakistan.[83] These gradually developed into the Indus Valley Civilisation,[84][83] the first urban culture in South Asia,[85] which flourished during 2500–1900 BCE in Pakistan and western India.[86] Centred around cities such as Mohenjo-daro, Harappa, Dholavira, and Kalibangan, and relying on varied forms of subsistence, the civilisation engaged robustly in crafts production and wide-ranging trade.[85]

During the period 2000–500 BCE, many regions of the subcontinent transitioned from the Chalcolithic cultures to the Iron Age ones.[87] The Vedas, the oldest scriptures associated with Hinduism,[88] were composed during this period,[89] and historians have analysed these to posit a Vedic culture in the Punjab region and the upper Gangetic Plain.[87] Most historians also consider this period to have encompassed several waves of Indo-Aryan migration into the subcontinent from the north-west.[88] The caste system, which created a hierarchy of priests, warriors, and free peasants, but which excluded indigenous peoples by labelling their occupations impure, arose during this period.[90] On the Deccan Plateau, archaeological evidence from this period suggests the existence of a chiefdom stage of political organisation.[87] In South India, a progression to sedentary life is indicated by the large number of megalithic monuments dating from this period,[91] as well as by nearby traces of agriculture, irrigation tanks, and craft traditions.[91]


Cave 26 of the rock-cut Ajanta Caves
In the late Vedic period, around the 6th century BCE, the small states and chiefdoms of the Ganges Plain and the north-western regions had consolidated into 16 major oligarchies and monarchies that were known as the mahajanapadas.[92][93] The emerging urbanisation gave rise to non-Vedic religious movements, two of which became independent religions. Jainism came into prominence during the life of its exemplar, Mahavira.[94] Buddhism, based on the teachings of Gautama Buddha, attracted followers from all social classes excepting the middle class; chronicling the life of the Buddha was central to the beginnings of recorded history in India.[95][96][97] In an age of increasing urban wealth, both religions held up renunciation as an ideal,[98] and both established long-lasting monastic traditions. Politically, by the 3rd century BCE, the kingdom of Magadha had annexed or reduced other states to emerge as the Mauryan Empire.[99] The empire was once thought to have controlled most of the subcontinent except the far south, but its core regions are now thought to have been separated by large autonomous areas.[100][101] The Mauryan kings are known as much for their empire-building and determined management of public life as for Ashoka's renunciation of militarism and far-flung advocacy of the Buddhist dhamma.[102][103]"""

prompt_query = prompt%text
output = model.generate(prompt_query)

print(output)

After Few minutes, it gives output like this with only CPU RAM :

\n\nThe text describes the history of ancient India from around 55,000 years ago to around 300 CE. It discusses the arrival of early modern humans in South Asia, the development of food crops and animal domestication, and the rise of urban civilizations such as the Indus Valley Civilization. The text also describes the emergence of Hinduism and Jainism, two non-Vedic religions that gained popularity during this period, as well as the political developments in ancient India, including the rise of monarchies and empires such as the Mauryan Empire."\nThe text does not provide a specific date for when these events occurred but instead uses language such as "around 55,000 years ago" or "around 300 CE" to indicate a broad time period.

Conclusion

In this blog post, we have shown how to use the Python bindings for GPT4all to generate text from the Large language models. We have covered the basics of using the bindings, as well as more advanced topics such as using prompts and generating different creative text formats.

GPT4all is a powerful tool that can be used for a variety of tasks, such as generating creative content, translating languages, and writing different kinds of informative text. The Python bindings make it easy to use GPT4all in your own Python code, which opens up even more possibilities. In next upcoming Blog, we will see how to use and integrate with langchain and create applications which will scale and useful in day-to-day tasks.