Common Corpus is the largest open and permissible licensed text dataset, comprising over 2 trillion tokens (2,003,039,184,047 tokens). It is a diverse dataset, consisting of books, newspapers, scientific articles, government and legal documents, code, and more.

• Announcing Common Corpus: A 2+ trillion token dataset that's fully open and accessible @ moz://a.

A NEW ERA OF OPEN LARGE-SCALE MULTI-MODAL DATASETS | LAION.

We present a dataset of 5,85 billion CLIP-filtered image-text pairs, 14x bigger than LAION-400M, previously the biggest openly accessible image-text dataset in the world

The RedPajama-Data repository contains code for preparing large datasets for training large language models.
RedPajama-Data: An Open Source Recipe to Reproduce LLaMA training dataset.

Inside Kaggle you’ll find all the code & data you need to do your data science work. Use over 50,000 public datasets and 400,000 public notebooks to conquer any analysis in no time.

Keshif is a web-based tool that lets you browse and understand datasets easily.

• Keshif @ GitHub