A feature checklist is close to useless as an evaluation tool, because every vendor on your shortlist will check every box. "Job costing" is a yes from all of them. So is scheduling, so is quoting, so is quality.
The questions below are the ones where answers actually differ. Several of them are uncomfortable to ask and that is the point — a vendor's reaction to an awkward question is itself information.
Ask these of every vendor you evaluate, including us.
On architecture
1Is the hour an operator clocks the same database record as the cost that lands on the job, or a copy that something moved?
If something moves it, ask what moves it, on what interval, and who owns the field mapping. A perfectly good answer exists here — some shops are happy with a maintained connector — but you need to know which architecture you are buying, because it determines who gets the support ticket when the numbers disagree. Why that seam costs what it does is worth ten minutes before any demo.
2Show me estimated versus actual on a job that is running right now, broken out by line — material, setup, run, outside service.
Make them do it live, on a job in progress, not a closed one. Many systems can produce this at close and cannot produce it in flight. Those are different products.
3How many screens to answer "where is job 4471 and will it ship on time"?
Ask them to do it. Count.
4What happens to my data if I leave?
Ask for the export format, specifically. "We can export" is not an answer; CSV of every table with keys intact is an answer. A vendor who is vague here is telling you something.
On the implementation
5Who does the implementation — you, or a reseller?
Neither answer is wrong, but they are different purchases. If it is a reseller, you are evaluating two companies and your contract is probably with only one of them.
6What is the total first-year cost including implementation, data migration, training, and any module I will need in month four?
Ask for it in writing as a single number. Then ask which line items are estimates rather than fixed.
7What is the longest implementation you have done in a shop my size, and why did it take that long?
The second half is the real question. Every vendor has a fast average and a slow outlier. The outlier is where you learn what goes wrong.
8What do I have to have ready before day one, and what happens if it is not clean?
Routings, part masters, customer records, open jobs, work-in-process balances. Ask whether you hand over exports and a description of the shop, or whether you are the one reconciling three spellings of the same customer. The answers differ enormously, and it is the difference between a week and a quarter — see how switching works here.
On the things that fail
9What is the most common reason an implementation of yours fails or stalls?
Ask it exactly that way. A vendor who says it has never happened is either new or not being straight with you. A vendor who names the real reason — usually floor adoption, sometimes a data problem nobody scoped — is worth more of your time.
10What does the floor actually have to do differently, and what happens on day three when somebody stops clocking?
Every costing system in existence depends on accurate floor data. Ask how the system behaves when it does not get it: does it fail visibly, or does it keep producing confident numbers that are wrong? Visible failure is the feature you want.
11Who answers the phone, how fast, and is it someone who has run a shop?
Ask for the actual response-time commitment and ask whether support is in your time zone during your second shift.
On the vendor
12Who is on your roadmap next quarter, and what did you ship last quarter?
You are buying a relationship with a company, not a snapshot of a product. A vendor who cannot tell you what they shipped in the last ninety days is either not shipping or not organized enough to know.
Three answers that should slow you down
"That is on the roadmap." Fine, if you are told the quarter and it is in writing in the contract. Otherwise price the system as though that capability does not exist, because for your purposes it does not.
"Every shop is different, so we would need to scope that." Sometimes true. But used in response to a basic architecture question, it is usually a way of not answering. Question one has a yes-or-no answer.
A demo on canned data that avoids your parts. Ask to see the demo run on a part configured like the work you actually take — your materials, your quantities, your routings, your outside processes. A walkthrough built on a vendor's tidy sample part tells you the software runs. It does not tell you how it behaves on a one-off in 17-4 with a tolerance problem.
Two things worth asking yourself, not the vendor
What is the thing that will actually break this? In most shops it is not the software. It is that the floor does not adopt the data collection, so the costing is garbage, so nobody trusts the reports, so everyone goes back to the spreadsheet. Whatever you buy, that risk is yours and it is the one worth planning for.
What is not on the invoice? A stack assembled from separate products carries a seam somebody has to own — the field mapping, the release notes, the drift — and that owner is almost always the person who can least spare the time. It never appears as a line item on any of the quotes you are comparing. Price it before you compare license fees, because it is usually where the real difference between two bids sits.
A note on who wrote this
This is published by a vendor, so read it accordingly. The questions above are weighted toward architecture rather than feature counts, because architecture is where we think the decision actually lives — a feature list can be matched in a release, and the answer to question one cannot.
Ask us all twelve. Ask the same twelve of everyone else on your list. The spread in the answers will tell you more than any of the individual answers will.
KG Ai ERP runs quoting, production scheduling, shop floor data collection, live job costing, suppliers, quality, CRM and a general ledger on one record. Built inside a working precision machining shop. Request a demo — thirty minutes, RFQ to invoice, on a shop set up like yours.