My Simple AI Research Request Gets An F On Report Card; 18 0f 19 Wrong
May 8, 2023 by Dave Haynes
Much has been written and discussed about the potential and use of AI for business and marketing, but far less has been written covering off its flaws – like accuracy.
I was putzing around in OpenAI this morning, trying different things. I tried to get a word cloud of the most common terms used by digital signage software companies, and I got back a broken image.
Then I tried asking this …
Prompt: List by head office location city and state the digital signage software companies in the United States
The response:
Here is a list of digital signage software companies in the United States, organized by head office location city and state:
- New York, NY
- Times Square Digital – (can’t find a company by that name, but if there is one, it would make sense that NYC is home)
- Four Winds Interactive (based very much in Denver, and company changed its name to Poppulo)
- Mvix – (based in DC area)
- Signpost Solutions – (it appears they are in the UK)
- Boston, MA
- Enplug – (they were in LA, but part of Spectrio now, so Tampa)
- FWI Boston – (based very much in Denver, the company changed its name to Poppulo)
- Scala – (based in Philly suburbs)
- Atlanta, GA
- Visix – we have a winner!!!
- Navori – (head office is Switzerland and business office over here is in Montreal)
- Houston, TX
- Rise Vision – (nope, and company mostly virtual)
- Chicago, IL
- Industry Weapon – (they were in Pittsburgh, but part of Spectrio now, so Tampa)
- Skykit – (Minneapolis)
- Los Angeles, CA
- NoviSign – (Israeli company)
- ViewSonic – (Taiwan, and they’re mainly hardware)
- San Francisco, CA
- Appspace – (Dallas, and no SF office even listed!)
- Scala – (Philly suburbs)
- Seattle, WA
- TelemetryTV – (Vancouver, BC … so at least driving distance!)
- Minneapolis, MN
- StrandVision – (have not even seen that company name in at least a decade … office is Eau Claire, Wisconsin, which has Minneapolis as the closest big city, I suppose)
- Salt Lake City, UT
- AxisTV Signage Suite – (that’s a product, not a company, and the company is Visix in Atlanta)
Note that this list may not be comprehensive, and some companies may have multiple office locations.
Note that this list may also not be even close to accurate. Of 19 entries returned, 18 were wrong! It also only lists 10 cities, and there are many more companies in many more cities … but might in part be a prompt thing. New York, off top of head, is main office location for ComQi, Videri, Zignage and probably a few more slipping my mind).
So why am I noting this? All the buzz and all the apps that have come online in 2023 suggest AI tools will make routine work easy and dizzyingly fast. I have tried generating test posts and ideas lists using ChatGPT and some of it has been pretty good. But my efforts to use this from even very basic research have been less than encouraging.
It will get better, but anyone in out ecosystem right now who sees AI tools as shortcuts to doing things like basic industry analysis and characterization should be aware that much of what they might get back from a prompt could (or will) be wrong. In the case of my simple request, get an F for a report card mark.
To quote Crocadile Dundee : “Yup, that’s what I saw!”
Good to hear Dave. Confirmation of same quality results in the self service kiosk space. I thought I was reading some synopsis from India datamart internet scraper. The good news is that the dumb people will be the first to rely on it, and will become even dumber?
Sadly, despite the issues, your results are probably on a par to what the report factories are charging big bucks for.
For reference I use grammarly to doublecheck my writing. After integrating AI I have seen some helpful suggestions on syntax. I have a tendency to ramble on with long sentences with lots of big words. Not the most effective.