BookCorpus - dataset

Transparency, accountability 🙈

Here are some key transparency limitations of the BookCorpus large language dataset:

• Source selection. Limited information is available about how books were selected for inclusion in the corpus, potentially introducing selection bias.

• Copyright and permissions. It is unclear whether proper permissions were obtained for all books included, raising legal and ethical concerns.

• Data cleaning process. Details about any preprocessing or cleaning steps applied to the text are not fully disclosed.

• Demographic representation. There is limited transparency regarding the diversity of authors and perspectives represented in the corpus.

• Language variety. While primarily English, the extent of dialect variation or inclusion of non-English text is not clearly specified.

• Versioning and updates. Clear information about different versions of the dataset and any updates made over time are limited.

• Quality control. Details about measures taken to ensure the quality and accuracy of the digitized text are not fully transparent.

• Metadata availability. The extent and accuracy of metadata (e.g., author information, publication details) accompanying the text is unclear.

• Duplicate content. Information about how potential duplicates or near-duplicates are handled within the corpus are lacking.

• Ethical review. There is no information regarding any ethical review process conducted during the dataset's creation.

Risks, harms 🛑

BookCorpus has been accused of illegally and unethically violating copyright, undermining authors' ability to control distribution of their creative works, which is essential for their livelihood, exhibiting racial and religious bias, misleading marketing and poor transparency.

Research, advocacy 🧮

• Bandy J., (2021). BookCorpus datacard/datasheet

• Bandy J., Vincent N. (2021). Addressing "Documentation Debt" in Machine Learning Research: A Retrospective Datasheet for BookCorpus
  
  

Investigations, assessments, audits 👁️

• Bandy J., Towards Data Science (2021). Dirty Secrets of BookCorpus, a Key Dataset in Machine Learning

News, commentary, analysis 🗞️

• https://towardsdatascience.com/dirty-secrets-of-bookcorpus-a-key-dataset-in-machine-learning-6ee2927e8650

• https://www.theguardian.com/books/2016/sep/28/google-swallows-11000-novels-to-improve-ais-conversation

• https://jack-clark.net/2021/05/17/import-ai-249-ibms-massive-code-dataset-dataset-archaeology-bookcorpus-facebook-wants-computers-to-read-the-world/

• https://info.deeplearning.ai/the-batch-face-recognition-for-the-masses-labeling-libel-documenting-datasets-what-machines-want-to-see-1