From scratch is a repository for:

• Updating our ancient programming skills to be useful for coding in python (3.7)
• Discovering the most awkward books in python's library
• Learning to deal with pycharm (2020.2)
• Scratching our pentesting itches
• Shaking down django (3.1.2) and PostgreSQL by creating a dashboard for some pentesting scripts
• Grokking git and github and tracking each others blundering about with a grin on our faces

The one not to scrap is our first engine with which we intend to explore:

• Searching across documents.
• Content suggestions using relevancy-ranked list of results.
• Relevancy ranking, which can also be used for personalisation of results and adding preferences (boosting the most common query completions).

… And a lot of data consists of unstructured, natural language content. By previous pigeon-holing of content, searching and combining of items can be a very expensive undertaking. Does not have to be. A non-exhaustive growing list of possibly interesting use cases for support beyond the usual:

• Identifying, collecting and producing electronically stored information in response to a request (GDPR compliance, litigation research) on an intranet.
• Identifying files that can be deleted or archived (Storage analytics).
• Combined with nlp capabilities for identifying what the question is, a search engine with stored knowledge graphs can answer natural language questions.
• …