AI in healthcare, finance, law, education and retail
The same tools show up in every industry. What differs is the cost of a mistake, the rules, and who must stay responsible.
Words on this page (2)
- tool
- : A function the model can ask your program to run, described by a name, description and argument schema.
- prompt
- : The input text you give a model, including instructions, examples and data.
What it is
Each field picks the tasks that are frequent, data-rich and checkable, and keeps people in charge of decisions that carry risk.
Healthcare: reading scans, writing visit notes, finding patients at risk. Regulators treat many of these as medical devices and expect evidence. Finance: fraud detection, credit scoring, document reading, customer chat. Decisions about credit must be explainable to the customer, and rules limit unfair bias. Law: searching cases and drafting. Lawyers remain responsible for every filing. Education: tutoring, feedback on drafts, lesson planning. The aim is learning, not finished homework. Retail: demand forecasts, recommendations, price tests, support chat.
A worked example: choosing the control
Rate each task on two lines: cost of a mistake, and how easily a person can check it. Summarizing a meeting: low cost, easy to check, so automate and sample. Drafting a contract clause: medium cost, checkable by a lawyer, so AI drafts and a lawyer approves. Approving a loan: high cost, affects a person's life, so a model may assist but rules, explanations and a human appeal must exist. The more a mistake costs, the more control stays with a person.
What goes wrong
Bias in data becomes bias in decisions. In 2018 Reuters reported that Amazon dropped a hiring trained on past resumes, because it had learned to mark down resumes that mentioned women's groups. In 2023 two New York lawyers and their firm were fined $5,000 after submitting a brief with court cases a chatbot had invented. Neither failure was about intelligence. Amazon's team tested the tool, found it was biased and shut the project down. The lawyers did not check their citations.
How a bias check works
Take 1,000 loan applicants in each of two groups. The model approves 600 in group A (60 percent) and 420 in group B (42 percent). The ratio is 42 / 60 = 0.70. A gap this size does not prove unfairness, since the groups may differ in income or debts. It is a to look closer. Compare applicants with similar finances: if group B is still approved less often at the same income and debt, the model is treating them differently and the data needs inspection. Ask which inputs act as stand-ins for the group, such as a postcode.
Explaining one decision
In many countries, including the US, a person denied credit is owed reasons. With a simple scoring model each input adds or subtracts points, so the biggest negatives become the explanation. Invented example: debt ratio -40, two late payments -25, short history -5. The reply says the main factors were debt and late payments. Models that cannot produce reasons like this are hard to use in lending.
Why human review fails
Putting a person after the AI does not guarantee checking. People skim when the tool is usually right. Say an assistant drafts 1,000 summaries and 50 are wrong. A reviewer who catches 60 percent lets 50 × 0.4 = 20 errors through. To test, secretly seed the queue with 20 known errors and count how many get caught. If the catch rate is low, change the process: show sources next to the draft, ask the reviewer to confirm specific facts, and rotate the work.
Common mistake
Copying a success from another industry without its safeguards. A tool that is fine for retail support is not automatically fine for medicine.
A small law firm uses an assistant to draft a research memo. The draft cites six cases. A junior associate looks up each one and finds that two do not exist. The partner adds a rule: every citation gets opened and read by a person before anything leaves the firm. The tool still saves hours, and the rule keeps the firm out of the news.
Check yourself
Which factor should set how much human control a task needs?
Not quite. High-cost, hard-to-check decisions need a person in charge.
Why did Amazon drop its resume screening tool?
Not quite. Learning from biased history teaches the model the same bias.
What went wrong in the court brief case?
Not quite. The tool made up cases and no person verified them before filing.
You got 0 of 3 right.
Go deeper
- ArticleAI-enabled medical devicesArticle
US FDA · The official list of AI devices cleared in the United States.
- ArticleStanford AI IndexArticle
Stanford HAI · Data on adoption by industry.
- ArticleAI Risk Management FrameworkArticle
NIST · A US government framework for judging and managing AI risk.
Spotted a mistake or something unclear? Report it on GitHub.