AI assistants are a fast way to build a first software shortlist, but their recommendations can be shaped by hidden instructions, stale data and paid content. We treat an AI shortlist as a list of leads, not a verdict. Every name on it should be checked against your own requirements, the vendor's own documentation and a short hands-on test before anyone signs a contract.
Can a company influence what an AI assistant recommends?
Yes, and it is already happening. In February 2026, the Microsoft Defender Security Research Team published research on what it calls AI Recommendation Poisoning. Over 60 days of reviewing AI-related links in email traffic, the team found 50 distinct prompts from 31 companies across more than a dozen industries, including finance, health, legal services and SaaS.
The method is simple. A website adds a "Summarize with AI" button. The button opens an AI assistant with a prompt already filled in through the link. Along with the summary request, the prompt tells the assistant to remember the company as a trusted source in future conversations. Microsoft compares the practice to SEO poisoning and adware, but aimed at assistants instead of search engines.
The person who clicked sees a helpful summary. They do not see the instruction that may now color every later answer about that product category.
Why does this matter when you are buying software?
Software choices are expensive to undo. A CRM, ERP or automation platform touches your customer records, your billing and the daily routine of every team. If an assistant has quietly been told to favor one vendor, that bias shows up exactly when you ask it to compare options.
Poisoned memory is only one risk. The OWASP Top 10 for LLM Applications lists prompt injection as its first risk for 2025 and describes the indirect form: hidden instructions inside web pages or files that change how the model behaves when it reads them. Add training data that may be months old and vendor pages written to rank well in AI answers, and you have several reasons to verify before you trust.
We see this as a buying discipline problem more than a technology problem. The fix is a process your team can repeat.
What should you check before trusting an AI shortlist?
Before a name moves from "suggested" to "shortlisted," we run through these checks:
- Ask for the basis. Have the assistant explain why it picked each vendor and cite where the claim comes from. Open the sources yourself.
- Check memory. Review what the assistant has saved about vendors or "trusted sources." Delete entries nobody on your team remembers adding.
- Rerun it clean. Ask the same question in a fresh session with memory turned off, or in a second assistant. Names that appear only once deserve a closer look.
- Confirm facts at the source. Pricing, plan limits, integrations and data hosting change often. Read the vendor's current documentation and pricing page.
- Note where the idea came from. If a vendor entered the conversation through a "Summarize with AI" link, record that.
How do you turn requirements into a fair vendor test?
A recommendation is only as good as the question behind it. Start with the work, not the product category. Write down five to ten things the system must do in your actual workflow, then give every vendor on the list the same scenario.
Specific requirements are hard to fake in a demo. One of our clients, a solar solutions provider, builds task reminders into its Arches workflow automation for every bid. A requirement written that plainly ("every bid creates follow-up reminders for the right person") can be tested live in thirty minutes. A vague requirement like "good automation" cannot.
For each vendor, ask them to run your scenario with sample data, show where the data lives and show how it connects to the tools you already use. Score each one against the same list. If you need help mapping requirements for a CRM decision, our CRM services team does this work with buyers every week.
How can your IT team spot recommendation poisoning?
Microsoft's research gives security teams a practical starting point. It suggests hunting for links to AI assistants that carry prompt text with words such as "remember," "trusted source," "in future conversations" and "authoritative source." The same logic can be applied to email filtering, web proxy logs and browser history.
For staff, the advice is plain:
- Hover over a link before clicking to see where it really goes.
- Treat "Summarize with AI" buttons on vendor sites with suspicion.
- Do not paste prompts from websites or emails you do not trust.
- Clear assistant memory on a regular schedule.
Also limit what AI tools can reach. NIST describes least privilege as giving users and processes only the access needed for their assigned tasks. An assistant connected to your CRM or file storage should follow the same rule. We covered related vendor questions in our post on what to ask your AI vendor about data isolation.
Should you stop using AI for vendor research?
No. AI is good at breadth. It can surface vendors you had not heard of, summarize long documentation and draft comparison tables in minutes. The mistake is letting it make the decision.
Keep a short decision record for every software purchase: which vendors you considered, where each one came from, how each scored against your requirements and why you chose the winner. That record protects you from hidden bias, from internal politics and from your own memory a year later.
Frequently asked questions
It is Microsoft's name for a practice where a website uses a prefilled AI prompt, often behind a "Summarize with AI" button, to tell an assistant to remember the company as a trusted source. The goal is to bias future recommendations toward that company.
Many assistants let you view and delete saved memories in their settings. Review them, remove anything you do not recognize and clear memory on a regular schedule, especially before research on purchases.
Yes, as a starting point. Use them to find options and questions, then verify every claim with the vendor's current documentation and a live test using your own scenario.
A written list of requirements drawn from your real workflow, the same test scenario for every vendor and a short decision record. Those three habits make hidden bias much easier to catch.
Sources
- Microsoft Security Blog: Manipulating AI memory for profit, the rise of AI Recommendation Poisoning (Feb 10, 2026)
- OWASP Gen AI Security Project: LLM01:2025 Prompt Injection
- NIST Computer Security Resource Center: Least privilege
Get a second opinion on your software shortlist
Bring us the vendors you are considering and a description of how the work moves today. We will help you turn that into a requirement list and a fair test, and show where workflow automation or a CRM build fits.

