The steps for building a documentary archive

Building a documentary archive requires developing a clear, formalized search and collection strategy, directly tied to the strategic question (or questions) the intelligence monitoring is meant to answer.

This is a process made up of several key steps, each requiring care and method. Here is a guide to navigating these steps and building a structured, relevant documentary archive.

1. Determine the search scope

The first step is to clearly define the subject of your intelligence monitoring. This means specifying the field to investigate, the specific questions you want to answer, and the goals of your research. This essential step lets you target subsequent searches more effectively and avoid collecting superfluous information.

2. Select the relevant sources

Not all information sources are equal. And a single information source is rarely enough. That is in fact why, since its founding in 2004, TKM has always prioritized the ability to collect and analyze a very wide variety of information sources in developing its software platform: patents, scientific publications, the web, theses, conferences, collaborative projects, start-ups…

It is therefore essential to identify the most relevant ones before embarking on a state-of-the-art study or configuring periodic intelligence monitoring. Some databases are free, others are paid. Costs can quickly become a limiting factor. That’s why TKM also offers privileged access to its own Data Lake, a global database of patents, scientific articles, projects and start-ups.

3. Set up a collection strategy

Once the search scope is known and the sources identified, it’s time to list the keywords that will enable exhaustive, relevant collection. Each database has its own “grammar”, and it’s worth getting to know it.

However, in most cases Boolean operators (AND, OR, NOT), truncation operators, or even proximity operators (NEAR, W..) will help you refine sets of keywords that combine effectiveness and exhaustiveness. Ideally, you should tend to keep a bit of noise (10 to 15%) rather than risk missing useful information.

But if this exercise becomes complicated, at TKM our team of analysts, made up exclusively of top-level scientists, has developed genuine expertise on the subject over the years.

4. Organize and index the documents

Nowadays, whatever the subject, collection usually results in a substantial volume of documents (anywhere from several hundred to several thousand, or even tens of thousands). To begin with, and before the analysis phase, a careful curation, enrichment and organization phase is essential.

This can involve adding useful metadata, extracting named entities relevant to your research, or standardizing affiliations (organization names) and semantically indexing the documents.

Then, setting up a classification system, or tags or a selection process, will be useful for making later searches easier: your own, but also those of the client teams down the line — management, R&D, IP, innovation, corporate venture, etc.

5. Extract useful information

Simply collecting information is not enough. But if your documentary archive contains several hundred documents, you probably won’t want to read all of them… and in fact that’s not even desirable.

Semantic segmentation, search, weak-signal detection and “outlier”-detection tools will be extremely useful for:

• confirming your view of the field (what you already knew),
• but also detecting weak signals (an emerging technology, a start-up, a new patent, a competitor’s strategic pivot…).

So by segmenting your documentary archive, and without having to read everything, you can, within a few hours, build strategic intelligence on the subject, launch an internal collective-intelligence effort and so feed your organization’s strategic steering.

And in the end, a few particularly inspiring or strategic documents (opportunity or threat) will warrant in-depth reading.

The purpose of the documentary archive is to let you address your strategic question at the overall level while still being able to move back and forth to individual documents.

The strategic intelligence you then build takes on a different dimension, and never leaves your internal stakeholders or your management indifferent. The goal is indeed to turn raw data into knowledge that is useful for decision-making.

6. Feed and update your archive

A documentary archive is a living tool that may need constant updating and enrichment to stay relevant. Even in the case of a one-off monitoring project (mapping, state of the art, landscaping…), it’s quite common for the decision to be made to turn it into periodic monitoring.

The pace of these updates will depend on the strategic intensity of the question, but also on how fast the field is moving technologically. So monitoring on AI or cosmetics requires at least monthly updates, given the volume of knowledge produced by research and industry players.

For this reason, among others, it is essential to keep a precise, formal record of the search strategies used.

And then to apply them regularly to the various sources, unless you have access to a global documentary archive such as TKM’s “Data Lake”, through which updates can be easily automated.

You will also need to run the new data through the various steps just outlined in this article. This work can prove tedious and can quickly turn into a nightmare for the analyst doing the monitoring. But some tools can save you precious time.