Reporting Systems and Dashboards: Choosing Between a BI Tool and a Custom Build
At the start of every month, someone spends several days pasting together exports from a handful of systems, fixing formats, reconciling figures, and turning it into a deck. Everyone looks at it in the meeting, nods, and then nobody changes a single decision because of it.
This is the most common starting point for a reporting requirement. The problem is not an absence of numbers. It is that the numbers arrive too slowly, the definitions do not agree, and nobody knows what to do after reading them.
Solving that is mostly not a technology-selection exercise. Technology selection is the last step. Handle the earlier ones and the choice of tool tends to reveal itself.
Start by scoping: who looks, at what, and how often
Before debating whether to build a dashboard, fill in the three columns below. You will immediately find that half the requirements need no development at all.
Who looks. Executives, department heads, and frontline staff want completely different things. Executives want a handful of trends. Managers need to drill down to the reason. Frontline staff want to know which specific items to deal with today. Mix all three roles into one screen and the usual result is that everyone finds it simultaneously overwhelming and insufficient.
At what. For each role, write down the action they might take after looking. If a metric would not change any behavior regardless of whether it goes up or down, it does not belong on the main screen. This step filters out vanity metrics better than anything else, and the logic is the same one used for setting social media KPIs.
How often. Checked every morning, checked in a weekly meeting, or checked after the monthly close — the technical cost of those three differs enormously. Real-time updates sound appealing, but they raise the complexity and the cost structure of the entire data pipeline. If the numbers are genuinely only read in a weekly meeting, next-day refresh is plenty.
Complete that table, then talk about tools.
Weighing BI tools against custom dashboards
By BI tool here we mean off-the-shelf business intelligence software: you connect your data sources and build charts and reports by dragging and dropping. A custom dashboard is a screen developed specifically for your requirements.
| Dimension | Off-the-shelf BI tool | Custom dashboard |
|---|---|---|
| Time to launch | Faster, mostly configuration | Slower, requires development |
| Flexibility | Users can switch dimensions themselves | Fixed views; changes need development |
| Fit with existing systems | Usually a separate entry point | Can be embedded, sharing permissions and navigation |
| Unusual business logic | Complex rules often hit a wall | Can be implemented fully |
| Acting on what you see | Generally view-only | See it and act on it in place |
| Long-term cost shape | Ongoing license and seat fees | Up-front development plus maintenance |
| Skill requirement | Somebody must know the tool and maintain the data model | Users learn no new tool |
In practice the judgment reduces to a few questions:
- Do users need to explore? If they need to change the angle on the fly, a BI tool fits better.
- Does the report need to be tied to an action? If spotting an anomaly should let you click straight through and deal with it, custom fits better.
- How unusual is the calculation? If your gross margin, revenue sharing, or revenue recognition follows company-specific rules that also change, custom development — or at least a well-built data processing layer — is less likely to get stuck.
- Who maintains it? A BI tool needs someone inside the company tending the data model and the reports over time. Without that person it will be abandoned just the same.
Many companies end up mixing the two: the core fixed views are custom-built and embedded inside the existing admin system, while exploratory analysis goes to a BI tool. That division of labor is usually more practical than an either-or choice, and it connects naturally to the planning described in A Guide to Building Internal Admin Systems.
Data source integration: garbage in, garbage out
Most of the work in a reporting project is not drawing charts. It is getting the data into one place and making it trustworthy.
Typical data sits in places like these: orders from the website and e-commerce channels, settlement records from the payment provider, warehouse or logistics systems, membership and CRM data, advertising platforms, and a pile of spreadsheets living on one colleague’s computer. Connecting them means running into every problem covered in system integration and API development: each system has a different data format, a different refresh timing, and some offer no interface at all.
Before you start connecting, confirm a few things:
Whether the keys line up. Is the same customer identified by the same identifier across systems? If the only way to match is by name or phone number, duplicates and typos have to be handled first.
What the timestamps mean. Order time, payment time, shipping time, settlement time — different systems record different ones, and any cross-system aggregation has to specify which applies.
How returns, allowances, and cancellations are reflected. These reverse flows are the easiest to overlook and they bear directly on whether the revenue figure is correct.
Whether historical data moves with you. How far back, and whether the field definitions changed along the way. This has a significant effect on effort.
How missing values are handled. Left blank, filled with zero, or flagged as unknown? Those three choices produce three different-looking reports.
There is no shortcut through this stage. A report only reflects the quality of its sources; however polished the front end, it cannot repair what is missing underneath.
Metric definitions: the real hard problem
Inside a single company, the word revenue can mean several different calculations: with or without tax, with or without shipping, before or after returns, counted on the order date or the shipping date, with or without gifts and credits included. Every version has its own logic. The problem is that none of it is written down.
So a reporting project must produce a metric definition table, stating at least four things for every metric:
- A plain-language definition. What question this number answers.
- The calculation. What is included and what is excluded.
- The time basis. Which timestamp field governs it.
- The data source. Which table in which system it comes from.
This table is signed off by the business, not decided by engineers. The method mirrors How to Write Software Requirements: pin down the definition of each term first, then talk about screens.
One practical rule: keep only one official definition per metric. If the business insists on two calculations, give them two different names and present them separately. Never let the same word mean different things on different screens — that removes the basis on which the whole reporting system could be trusted.
Permissions and refresh frequency
Permissions should follow the role design your company already uses rather than introducing a second scheme. The usual structure has three layers: which metrics someone can see, which scope they can see (the whole company, a single store, their own results), and whether they can export. That third one matters most, because reports often contain customer lists and revenue detail, so export rights are effectively the gate on data leaving the building. For the underlying principles, see the treatment of permission tiers in Building a Membership and CRM System.
Refresh frequency is best set per screen rather than uniformly across the suite. Operational numbers people act on the same day, such as today’s orders or the pending queue, need to be closer to real time. Trend-level management metrics are usually fine with a next-day refresh. Every screen should state clearly what point in time the data is current as of; that one line saves an enormous amount of back-and-forth about why the figures differ from someone’s own calculation.
One more caution: website and marketing figures come from different sources than transaction systems, and they are defined differently. Adding numbers from an advertising platform or an analytics tool directly to order system figures is easy to mislead with. For how the two divide responsibilities, see A Practical Guide to GA4.
Phasing: get one chart genuinely used first
The classic failure of a reporting project is planning dozens of charts at once and then finding nobody looks at them. The sequence more likely to succeed goes like this:
- Phase one: one role, one subject. Pick the most painful one, usually revenue or orders, build only that line, and establish the data sources, the metric definitions, and the refresh mechanism along the way.
- Verify it is actually used. After launch, watch whether anybody checks it regularly and whether any decision changed as a result. If not, the thing to fix is the metrics and the screen, not the number of charts.
- Phase two: the ability to drill down. Let users click from a total into the detail and find the reason. This step is usually worth more than several extra charts.
- Phase three: extend to more roles and subjects. Adding other departments on top of the existing data foundation costs far less than phase one did.
Decide early who owns the metric definitions. Business rules change; new product lines and revised revenue-sharing arrangements both move the calculation logic. If nobody is responsible for keeping the definitions in sync, the dashboard will quietly start lying a few months later — and that failure is more dangerous than an outage, because nobody notices it.
The value of a reporting system lies not in how attractive the screens are but in whether anyone makes a different decision because of them. Sort out the roles, the metric definitions, and the data quality, and choosing the tool turns out to be the easy question.
If you are weighing a BI tool against a custom dashboard, or your current reports never reconcile, talk to NETVANA about your reporting requirements. Software services are quoted individually rather than sold as fixed packages, and consulting covers architecture review and technical due diligence; for what each service includes and delivers, see the software services overview.
Further reading: if your processes are still in spreadsheets and you are considering an upgrade, see A Guide to Building Internal Admin Systems. To connect data across systems, start with A Guide to System Integration and API Development. For how to read website and marketing numbers, see A Practical Guide to GA4. Because metrics slide into vanity so easily, see How to Set Social Media KPIs. And for which contract model suits a phased reporting rollout, see Fixed Price or Agile.
