We built an AI voice agent for a property manager to handle tenant maintenance calls.
Week two, a contractor calls the office. "Your AI quoted $300 for this disposal swap. Why are you only approving $250?"
We built an AI voice agent for a property manager to handle tenant maintenance calls. Seemed straightforward: tenant reports broken garbage disposal, AI triages it, dispatches contractor, everyone saves time.
Week two, a contractor calls the office. "Your AI quoted $300 for this disposal swap. Why are you only approving $250?"
We didn't see that coming. The AI was generating cost estimates as an internal sanity check, numbers that appeared on the contractor-facing dispatch. Transparent, sure. Also: instant negotiation anchor.
We pulled the estimates off the contractor view that afternoon. The AI still runs the numbers, but contractors see the work order, not the math. Internal tool, not public benchmark.
The broader point: every automated decision that touches money will surface unintended power dynamics. A chatbot that shows reasoning becomes a citation in an argument. A pricing model that shows confidence bands becomes evidence in a dispute. Transparency sounds neutral until someone uses it as a wedge.
If your AI touches vendor relationships or cost approvals, audit what the other side can see. Not for secrecy, just for clarity about what you're anchoring and what you're negotiating.
What automated outputs in your business could flip from helpful to adversarial if the wrong person saw them?