Choose the right task for your first trial

Ask your team to record recurring tasks for one week. For each task, assess the time spent, the information used, and the consequences of getting it wrong. A frequently repeated task whose output is easy to check and whose errors can be reversed can be a good starting point.

For example, you could consider generating draft replies to customer support messages. At the initial stage, the message is not sent to the customer automatically; an employee reviews the draft. This allows the business to measure response preparation time while preventing incorrect information from reaching the customer directly.

Prepare the data and define usage boundaries

Decide which information will be provided to the AI. Outdated price lists or conflicting procedures make it harder to generate accurate answers. First, review the documents that will be used and assign someone to take responsibility for the content of each one.

Before transferring customer information, employee records, or trade secrets to a tool, review its access, retention, and usage terms. If possible, run the initial trial using examples that contain no personal data. Put the answer to this question in writing: “What information may this system use, and what actions must it not perform?”

NIST’s AI Risk Management Framework provides a voluntary framework for evaluating AI systems in terms of their context of use and associated risks. For a small or medium-sized business, this means identifying the person responsible, the review method, and the conditions for stopping the system from the outset. NIST AI Risk Management Framework

Do not measure success by how impressive the tool looks

Measure the current situation before the pilot begins. For example, you could use the following suggested table:

Do not focus only on how quickly the first draft is generated. If a text that takes ten minutes to write can be drafted with AI in two minutes but then takes fifteen minutes to correct, the process may not be delivering any benefit.

Start with a small pilot

One suggested pilot plan is to prepare the examples first, then run a trial with a limited team and a specific task. Record accepted and rejected outputs throughout the process. Make sure there is a clear way to hand the task over to an employee when something unexpected happens.

At the end of the pilot, make one of three decisions: expand the scope, change the approach, or stop the implementation. Ending an unsuccessful trial early is also a valuable outcome; it prevents a larger investment from being based on a mistaken assumption.

Which tasks are better suited to a first project?

Looking only at how often tasks are repeated can be misleading when choosing them. A task performed many times a day may be unsuitable for an initial trial because of the high cost of errors. For example, approving a payment and drafting a product description do not require the same level of oversight. For the first project, choosing a task whose incorrect output can be easily spotted and corrected before it is used makes evaluation easier.

Consider a wholesale business. Suppose its sales team receives similar questions every day about product dimensions, package contents, and delivery options. The first trial could involve an assistant that drafts answers using the current catalog. The assistant does not change prices, promise delivery times to customers, or create orders. It only suggests information contained in approved documentation, and an employee checks the response before sending it.

In this example, the narrow task definition makes the results easier to examine. The draft is expected to refer to the correct product, avoid adding information that is not in the documentation, and state when an answer cannot be found. If the project progresses successfully, new product groups can be added. This allows the scope to grow without trying to build a system that takes over all customer service from day one.

Prepare test data that reflects the variety of real work

A test made up only of easy examples can give a misleading impression of success. For a system that will classify support messages, include incomplete and lengthy examples, messages covering multiple topics, and examples with spelling mistakes alongside clear questions. Try asking the same question in different ways. Situations frequently encountered at work should be represented in the evaluation set.

The person preparing the examples should also write down what information an acceptable answer must contain. There is not always a single correct sentence; the expectation may be that certain information is provided accurately and prohibited behavior is avoided. For example, in response to a delivery question, giving a date that has not been confirmed by the warehouse would be an error, while asking the customer for an order number could be an appropriate next step.

Whenever you notice a new problem during the trial, add it to your test list. This allows you to check whether an improvement fixes that example without causing problems in previous examples. A simple evaluation table may be enough for a small business to get started; what matters is applying the same criteria consistently.

Explain the team’s role from the outset

If employees do not know where the system is intended to be used, the same tool may be tried for very different purposes. One person might draft an email while another makes a price commitment to a customer. Bring together the pilot’s purpose, the permitted data, and the points requiring human review in a brief usage note. Have the people who will use the tool review this note together.

It is also useful to designate a process owner and a technical lead. The process owner monitors whether the task is completed correctly, while the technical lead handles access and connection issues. In small businesses, one person may take on more than one role, but responsibilities must remain clear. It should be known in advance who will investigate when an incorrect answer is reported.

Do not limit employee feedback to “Did you like it?” Ask when they had to rewrite a draft completely, which information they verified elsewhere, and which step saved them time. The details of daily work reveal limitations that may not be apparent in a short demonstration.

Include the review workload in your savings calculation

Consider a hypothetical calculation: suppose your team prepares 200 replies per month, with each reply taking an average of 10 minutes. If the combined preparation and review time falls to 6 minutes when using an AI draft, the example calculation produces a difference of 800 minutes. This figure is not a product promise; it illustrates how you can evaluate your own measurements.

Deduct the time spent updating documents, reviewing incorrect examples, and managing the system from this time saving. Record subscription, integration, and training costs separately as well. Decide how the time saved will be used in the business: to answer more customer inquiries, prepare quotes faster, or ease the workload during busy periods?

Not every gain will appear directly as a reduction in staffing costs. Preparing more consistent answers or making it easier for new employees to access information can also be valuable. However, define these benefits from the outset and track them using observable indicators. A vague assessment that “productivity has improved” is not enough to support the next investment decision.

When should the pilot be expanded, and when should it be stopped?

Before expanding the scope, consider checking whether different users achieve similar results, how the system behaves when content is updated, and whether errors are manageable. A successful initial demonstration is not sufficient evidence for everyday use. If results vary, first try to understand the source of the problem.

If errors affect important business decisions, review time cancels out the time saved, or the necessary documents cannot be kept organized, narrowing the scope may be more appropriate. Sometimes the right next step is to organize product information or simplify the workflow rather than choose a more advanced model. The initial project can also reveal gaps in the business’s preparation.

Make your first step a one-page task brief, not a list of tools

Write down which task will be supported, who will use the system, what data is needed, and how the outcome will be measured. This page will also guide you when speaking with providers. An investment in AI is worthwhile to the extent that it makes a tangible contribution to your business and can be managed by your team.