The challenge of technology monitoring now lies more in sorting and organizing available data and information than in searching for information.
Patents, scientific publications, research projects, institutional reports, web sources and industry news accumulate and spread at a rate that far exceeds human analytical capabilities.
Illustration in cosmetics
In the cosmetics industry, nearly 2,000 new patents are published worldwide each month, demonstrating constant innovation and an intense race for novelty.
Even more dramatically, artificial intelligence is experiencing an information explosion: the subject is the focus of more than 50,000 scientific articles every month.
This information overload, or "infobesity," makes it difficult to select, analyze, and prioritize relevant content. R&D and innovation teams find themselves facing a virtually uninterrupted flow of information.
In this context, the challenge is no longer simply access to information, but your ability to sort, structure, and analyze it. How can you leverage these massive flows of heterogeneous data without spending your days manually sorting it? How can you avoid both informational noise and blind spots?
Contrary to what the hype might suggest, generative AI has not overturned traditional intelligence approaches and is not about to do so for at least three reasons:
- Traditional methods yield excellent results in well-defined contexts and with structured data. To achieve similar performance, generative AI would require very large models or very expensive retraining.
- The results of generative AI are very difficult to interpret, unlike classical models which allow for true explainability.
- Classical models are less subject to variations and their results are more reliable, unlike generative AI whose robustness can be problematic (for example, because of hallucinations).
Generative AI thus provides many useful and interesting methodological tools. However, for reasons of efficiency on the one hand, and also due to its high environmental cost (a subject on which TKM is fully engaged), this contribution complements traditional approaches and does not replace them.
The solution developed by TKM is based on a rigorous approach, structured around a balanced mix combining, on the one hand, the use of AI when it is really useful and, on the other hand, data mining applied to technological literature.
It results from more than 20 years of recognized expertise combined with a genuine capacity for innovation and integration of emerging technologies, not for their novelty, but for their usefulness.
By relying on structured search strategies, expert systems and complemented, when necessary, by artificial intelligence building blocks, TKM's monitoring and analysis software transforms large volumes of raw data into information usable by R&D, Industrial Property and Innovation teams.
Understanding the fundamentals: data mining, the foundation of technology monitoring
Data mining, the backbone of large-scale intelligence gathering
For 25 years, data mining has established itself as the foundation of large-scale intelligence gathering, and this is no coincidence.
Data mining encompasses all the techniques used to explore, clean, and structure large volumes of data in order to extract useful information. In the context of competitive intelligence, this includes identifying trends, thematic groupings, emerging issues, or weak signals.
In our daily practice of technology monitoring for over 20 years, one observation consistently emerges: only a fraction of the information collected has real strategic value. Experience shows that only 10 to 15% of the documents collected are actually relevant for analysis and informing decision-making.
The role of data mining is precisely to manage this gap between abundance and utility. It allows us to:
- reduce informational noise without sacrificing comprehensiveness,
- eliminate duplicates,
- detect weak signals,
- structure and organize the data according to relevant criteria,
- prepare a usable corpus for analysis (human and computer-assisted)
Without this preliminary work, monitoring quickly becomes unmanageable, regardless of the power of the tools used. Robots and other crawlers quickly overwhelm you with masses of unmanageable information and will divert your R&D teams from a task that is nevertheless vital to their profession: staying constantly informed about the state of the art, the competition, and the freedom to operate!
→ Read also – Building a document repository: collecting data useful for technological or competitive intelligence
Expert systems: Business-driven intelligence
The information needs, the nature of the data processed and the purpose of an R&D team, an IP department or even an Innovation and open innovation oriented team are not the same.
The treatments that are useful for each of these needs are not always the same, and the workflows are sometimes very different.
The structuring of data for an aeronautics company probably has no connection with that useful to a big pharma or a precision mechanics SME.
Unlike magic tools and the "black box effect" that every self-respecting analyst detests, expert systems constitute the first step in the operational implementation of intelligence within the TKM platform . They are based on data mining principles, that is, on explicit rules that can be configured by each type of user: Boolean queries, proximity operators, business filters, thesauri and classification criteria, cleaning and normalization, etc.
This approach is particularly effective when the relevance criteria are clearly and easily identifiable: targeted technologies, types of documents, actors to monitor, geographical areas, application domains.
Expert systems then allow for a lower cost1 :
- to transform a business need into a formalized search strategy,
- to maintain complete control over the monitoring areas,
- to automate the sorting, classification and dissemination of information.
They offer an excellent compromise between performance, transparency and cost control, while remaining easily adjustable as needs change.
When data mining reaches its limits: the targeted contribution of precision AI
Not all data processing operations can be easily handled solely by explicit rules. When determining these rules becomes complex, then data mining reaches its limits.
Let's take an example!
Your business, your industrial sector or your own strategy is based on respecting environmental criteria in the design and manufacture of your products.
It is therefore legitimate for your monitoring to highlight as a priority and bring to the attention of your R&D teams any innovation, product or patent from a third party (a competitor or a startup) that would fall under these "environmentally friendly" criteria.
But how do you define, in a perfectly exhaustive way, all these criteria and translate them into a system of rules within your monitoring tool? This is likely to be complicated, or even totally impossible.
There are a thousand and one ways to describe this notion of eco-responsibility: biodegradable, chemical-free, less energy-intensive, rare earth-free, better CO2 balance, etc.
This is where TKM's precise and customized AI can play a crucial role, with levels of accuracy unmatched by data mining and/or generic AI. The models will then be trained on historical data annotated according to the specific use case required.
At TKM, we firmly believe that in these complex cases, AI offers a genuine complementary alternative to expert systems derived from data mining. It doesn't replace them, it enhances them.
But we are equally convinced that generic AI alone (ChatGPT being the prime example) and the magical promise of a model giving satisfactory results in three clicks and four likes, does not work!
This is why the precision AI designed and offered by TKM relies on processing based on both generic artificial intelligence models (classification, clustering, entity detection) and large-scale language models (LLMs), but which also and above all incorporates a preliminary personalization step. It is then that we can speak of precision AI (or personalized AI) applied to competitive intelligence.
Our AI is trained on specific use cases provided by users, in order to complement expert systems when they reach their limits.
TKM software: a platform designed for competitive intelligence data mining
A “Data Lake” designed for 360° exploration
At the heart of TKM Platform is a DataLake fed by an unparalleled diversity of sources:
- worldwide patents,
- scientific publications
- research projects and collaborative projects,
- clinical trials,
- web sources and industry news,
- global database of innovative startups and SMEs,
- global database of academic players and major groups.
Users can precisely target their monitoring by topic, source, author, organization, or time period. Specialized crawlers also allow users to track specific institutions or feeds, such as funding agencies, universities, or research centers.
This wealth of data is essential for deploying effective intelligence strategies and detecting weak signals at an early stage. This is achieved through a skillful combination of expert systems and the judicious use of AI.
Human support to structure the approach
The effectiveness of competitive intelligence relies on the performance of the tools, but even more so on the methodology. This is why TKM supports organizations from the initial stages: clarifying the issues, formalizing a competitive intelligence plan, and defining search strategies.
The objective is to translate a strategic problem into an operational system.
Depending on the needs, the monitoring teams can be autonomous or supported (on an ad hoc basis or over time) to adjust and enrich the rules, scopes and processes.
→ Read also – TKM: dual expertise to put AI at the service of Industrial Property
Rethinking technology monitoring in the era of AI
From raw query to structured information: data mining in action in TKM Platform
Step 1: Formulate a precise and comprehensive search strategy
It all starts with expressing the need.
On the TKM Platform, users are guided through the process of moving from natural language formulation to a structured search strategy: keyword prioritization, logical operators, and filtering criteria. This step is crucial to avoid two common pitfalls: overly broad searches that generate excessive noise, or overly restrictive searches that create areas of silence.
Step 2: Clean, enrich, and structure the collected data
Once the data is collected, data mining and AI come into play.
Filtering, classification, and tagging algorithms reduce the volume of irrelevant documents and structure content according to criteria defined with the client. If necessary, trained models will complement this step, where data mining reaches its limits.
Step 3: Organize the information for analysis and capitalization
The relevant information is then organized into thematic folders, populated automatically or manually. The classification criteria are fully customizable: theme, source, date, type of actor, etc.
This structure facilitates analysis, targeted dissemination, and the long-term capitalization of knowledge. In the case of competitive intelligence, one of the criteria for organizing data could be a list of competitors, categorized by type of actor or by geographic area.
The combined use of data mining and AI is a real game changer
Getting to the heart of the matter without losing sight of the big picture
Data mining allows for the rapid extraction of high-value content data . Advanced processing techniques also enable the production of syntheses, summaries, and answers to questions posed in natural language, all within a structured corpus.
Detecting implicit concepts and weak signals
When explicit rules are no longer sufficient, AI enhances data mining by detecting concepts not explicitly formulated. This capability is particularly useful for identifying weak signals or emerging trends at the intersection of multiple domains.
Structured monitoring over time
The intelligence projects are updated according to a frequency defined by the intelligence teams. Those receiving the intelligence have continuous access to structured, up-to-date information ready for immediate use. This use is sustainable because the capitalization of knowledge and interactions between teams on the data sets constitutes a true intangible asset within the company.
Concrete example: the use of data mining for collaborative monitoring
An industrial player in the agri-food sector wishes to monitor, on an international level and on a monthly basis, the scientific and technological literature (mainly patents, scientific articles and start-up news) around the subjects of fermentation.
The flow is approximately 800 new documents to be sorted per month .
Despite a carefully refined search strategy with the TKM teams, the volume of uninteresting documents remains high (our famous 10% rule) and requires prior sorting before implementing an automatic classification (by type of yeast and areas of application, in particular).
At 2 minutes per document, this theoretically represents 3 days of work , and that's just to clean up the incoming data! And this needs to be done every month…
That is almost a month and a half of work per year, for arduous work that does not create strategic added value.
And focusing on just one monitoring topic when this company needs to monitor at least half a dozen others…! It's not feasible.
In this specific case, fortunately, the criteria by which interesting information can be distinguished (business rules) are quite simple, as are the rules for classifying news in the internal knowledge and collaboration system.
With just a few days of preparatory work, a system based on these rules was able to be deployed with a level of accuracy deemed satisfactory by the manufacturer.
The solution, deployed in this way and based exclusively on data mining , provides a fully operational service with minimal implementation costs. The processing times for sorting, cleaning, enriching, and classifying data are on the order of a few seconds for each new piece of information. An alert system then allows R&D teams to be notified, as needed, of the news that is of most interest to them.
However, it may happen that a system based exclusively on business rules does not allow for satisfactory accuracy .
It is typically in this type of situation that the use of Machine Learning and the training of a precision AI should be considered in order to ultimately provide the same service (automatic and instantaneous processing) and usefully supply the company's R&D or IP teams.
Conclusion
Effective technology monitoring relies first and foremost on the judicious, structured use of the right data mining tools, driven by the business and fueled by human expertise in monitoring.
Without this backbone, neither AI nor human expertise can produce reliable and actionable analyses.
By combining data mining, expert systems and targeted contributions from artificial intelligence, TKM offers a pragmatic and sustainable approach to technology monitoring, aligned with the real challenges of innovative organizations.
Do your teams need more readable, better structured and truly actionable intelligence? Contact the TKM team to discover how to put data mining and personalized AI at the service of your innovation decisions.
2. Both economically and environmentally.
