Thump and Flow: Conceptual Workflow Logic for Scheduled vs. On-Demand Clinical Integration

This overview reflects widely shared professional practices as of April 2026; verify critical details against current official guidance where applicable. Clinical integration is not a one-size-fits-all endeavor. The decision between scheduled batch processes and on-demand event-driven flows shapes the entire architecture, affecting data latency, system resilience, and maintenance burden.

The Core Problem: Timing in Clinical Integration

Healthcare integration platforms handle vast amounts of data—lab results, orders, clinical notes, and more—each with its own urgency. A scheduled integration (thump) pushes data in bulk at fixed intervals, while an on-demand integration (flow) triggers immediate data exchange in response to events. The conceptual workflow logic differs fundamentally: thump relies on deterministic timing, flow on causality. Understanding when each is appropriate requires examining the nature of the data, the consuming system's tolerance for latency, and the operational cost of constant connectivity. For instance, a nightly batch that updates a data warehouse is well-suited for population health analytics, but the same approach would be problematic for real-time medication alerts.

The challenge is that many teams default to one pattern without fully analyzing trade-offs. In this guide, we break down the conceptual workflow logic of both scheduled and on-demand clinical integration, providing a decision framework that goes beyond simple latency arguments. We address common misconceptions, such as the belief that on-demand always implies real-time (it does not—there are degrees of urgency), and that scheduled batches are inherently simpler (they introduce their own complexities around conflict resolution and staleness). By the end, you will be equipped to map your clinical use cases to the appropriate integration rhythm.

Why 'Thump and Flow'?

The metaphor captures the contrasting cadences. 'Thump' is the rhythmic, predictable pulse of a scheduled batch—like a steady drumbeat. 'Flow' is the continuous, adaptive current of on-demand events—like water responding to terrain. Both can coexist in a well-designed integration architecture, but only when their conceptual foundations are understood.

Who Should Read This Guide

This article is for integration architects, clinical informaticists, healthcare IT managers, and anyone involved in designing or maintaining clinical data exchange. It assumes a basic familiarity with HL7, FHIR, and typical healthcare integration patterns but explains conceptual logic accessible to those new to the field.

Conceptual Foundations: Defining Thump and Flow

To design effective clinical integrations, we must first define the two paradigms at a conceptual level. 'Thump' refers to scheduled, batch-oriented data exchange where a process runs at a predetermined time or interval, collecting, transforming, and transmitting a set of data. The logic is time-triggered: 'at 2 AM, send all new lab results from the last 24 hours.' The 'flow' paradigm is event-triggered: 'when a lab result is finalized, immediately send it to the subscribing systems.' The conceptual difference is not merely about latency but about coupling and causality.

In a thump workflow, the sender and receiver are loosely coupled in time—the sender does not need to know when the receiver will pick up data, and the receiver does not need to be online when the data is sent. This decoupling simplifies error handling: if the receiver is down during a batch, the batch can be retried later. However, it introduces data staleness: between batches, the data in the consuming system is potentially outdated. In flow workflows, the sender and receiver are coupled in time—a successful event requires both parties to be available. This tight coupling can lead to complex retry and compensation logic when failures occur, but it ensures that data is fresh as soon as it is created.

Another key dimension is the volume of data. Thump workflows aggregate data over a period, making them efficient for high-volume, low-urgency data like nightly data warehouse loads. Flow workflows handle individual transactions, which is ideal for low-volume, high-urgency data like medication orders that need immediate verification. The challenge arises when a high-volume feed also requires low latency—this is where streaming architectures with buffering and backpressure come into play, blurring the line between thump and flow.

The Role of Data Freshness

Data freshness requirements vary by clinical use case. For a patient summary used in an emergency department, minutes matter. For a monthly population health report, hours are acceptable. Conceptual workflow logic must map these freshness tolerances to integration patterns. A common mistake is to demand real-time for all data, leading to unnecessary complexity and system overload.

Error Handling Paradigms

Batches naturally support 'fail and rerun' error handling: if a batch fails, you can fix the issue and rerun the entire batch. On-demand flows require 'fail and compensate' patterns: if an event fails to process, you may need to trigger compensating actions to maintain consistency. Understanding these paradigms is crucial for choosing the right approach.

Comparative Analysis: Thump vs. Flow in Clinical Contexts

This section provides a structured comparison of thump and flow across several dimensions relevant to clinical integration: latency, reliability, resource utilization, complexity, and use case fit. The goal is not to declare a winner but to map each approach's strengths and weaknesses.

Latency: Thump introduces deterministic latency equal to the batch interval. For a nightly batch, data can be up to 24 hours old. Flow introduces variable but typically low latency—from milliseconds to seconds—depending on the event processing pipeline. However, flow can experience latency spikes under heavy load if the system uses queuing with backpressure.

Reliability: Thump benefits from simplicity in error recovery: rerun the batch. Flow requires more sophisticated error handling, including dead-letter queues, retry policies, and idempotency guarantees. In clinical settings, where data loss is unacceptable, flow architectures must be carefully designed with exactly-once semantics.

Resource Utilization: Thump creates periodic spikes in CPU, memory, and network usage. This can be scheduled during off-peak hours to minimize impact. Flow creates a more constant but lower baseline load, which may require continuous scaling infrastructure. For on-premises systems with fixed capacity, thump may be more predictable.

Complexity: Thump is conceptually simpler but operationally tricky when conflicts arise (e.g., updates to the same record between batches). Flow is conceptually more complex but operationally more transparent because each event is processed individually. Teams often underestimate the hidden complexity of managing state consistency across multiple thump processes.

When to Choose Thump

Choose thump when the data is not time-sensitive, the volume is large, and the receiving system can tolerate staleness. Typical examples include data warehouses for population health, billing systems that run monthly cycles, and research databases where historical accuracy matters more than real-time updates. However, be aware that thump can mask integration failures: if a batch fails to run, the problem may go unnoticed until the next batch.

When to Choose Flow

Choose flow when data must be acted upon immediately—such as medication reconciliation, clinical decision support alerts, or real-time eligibility checks. Flow is also better when the data stream is unpredictable, with bursts that would make scheduled batches inefficient. A hybrid approach, where some data is batched and some flows, is common in practice.

Decision Framework: How to Choose Between Thump and Flow

This step-by-step framework helps integration teams systematically evaluate their use cases. Step 1: Identify the data consumer's latency tolerance. Step 2: Estimate data volume per unit time. Step 3: Assess system coupling constraints. Step 4: Evaluate error recovery capabilities. Step 5: Consider regulatory requirements (e.g., audit trails may require deterministic capture).

For example, consider a lab results integration. If the consuming system is an EHR that needs to display results to clinicians, the latency tolerance is low (minutes), volume is moderate (hundreds per hour), and the EHR should be available most of the time (coupling is acceptable). This suggests flow. If the consumer is a research database that only needs daily aggregates, thump is more efficient.

A common mistake is to force flow into a thump architecture by using polling. Polling (e.g., every 5 minutes) is a hybrid that mimics flow but introduces the complexity of both: it requires constant connectivity like flow but still has latency like thump. Avoid polling as a compromise; instead, design the integration to match the use case.

Step-by-Step Decision Process

1. Define latency requirement: Is 'immediate' truly required, or is 'within 15 minutes' acceptable? Many clinical scenarios can tolerate a few minutes of delay.
2. Quantify data volume: Measure peak and average transaction rates. High volume may favor batching to reduce overhead.
3. Assess infrastructure: On-premises systems may have fixed capacity, making thump more predictable. Cloud-native systems can scale for flow.
4. Evaluate error handling: If the business can tolerate reprocessing entire batches, thump is simpler. If individual events must be tracked, flow is needed.
5. Check regulatory constraints: Some regulations require immutable audit logs of data exchange. Thump naturally creates batch-level logs; flow requires per-event logging.

Scenario 1: Scheduled Batch for Population Health Analytics

An integrated delivery network (IDN) operates a population health analytics platform that aggregates data from multiple EHRs, billing systems, and patient registries. The analytics platform generates reports on chronic disease management, readmission rates, and preventive care gaps. These reports are used for quarterly reviews and quality improvement initiatives. The data does not need to be real-time; updates once a day are sufficient. The IDN chooses a nightly batch (thump) process that extracts all new and modified records from each source system, transforms them into a common schema, and loads them into the analytics database.

The workflow logic is straightforward: at 2 AM, a job runs on each source system that queries for records changed in the last 24 hours. The results are written to a staging area, transformed, and then inserted into the analytics database. Error handling involves retrying failed batches and sending alerts if the batch fails entirely. The IDN avoids on-demand integration because the analytics queries can be resource-intensive, and running them during the day could degrade EHR performance. The thump approach also simplifies auditing: each batch has a timestamp and a manifest of records, making it easy to verify completeness.

However, the IDN encountered a challenge: some source systems had overlapping patient identifiers, causing duplicate records when changes were made in different systems between batches. They resolved this by implementing a master patient index that was updated before the batch ran, but this added complexity. Despite this, the thump approach remained the right choice because the latency tolerance was low, and the volume was high (millions of records per night).

Lessons Learned

Key takeaway: Thump works well when you can tolerate staleness and have the operational capacity to manage batch schedules. However, be prepared for data consistency issues across multiple sources. A pre-batch deduplication step is often necessary.

Scenario 2: On-Demand Flow for Medication Reconciliation

A health system implements an enterprise medication reconciliation platform that must update the patient's medication list immediately when a new order is placed, a medication is administered, or a prescription is filled. Delays could lead to duplicate therapy, adverse drug interactions, or missed doses. The integration must be event-driven: each medication-related transaction triggers a flow that sends the updated medication list to all subscribing systems, including the EHR, pharmacy system, and clinical decision support (CDS) engine.

The workflow logic uses an enterprise service bus (ESB) that listens for HL7 messages (e.g., OMP for orders, RDE for dispense). Upon receiving a message, the ESB transforms it into a FHIR MedicationRequest resource and publishes it to a topic. Subscribers (EHR, pharmacy, CDS) consume the event in near-real-time. Error handling is critical: if the CDS engine is down during an event, the event is placed on a dead-letter queue and retried with exponential backoff. If the event cannot be processed after several retries, an alert is sent to the integration team.

The team initially considered a thump approach with a 5-minute polling interval, but they found that even 5 minutes of delay could lead to clinical risk. They also discovered that polling would add unnecessary load to the source systems, which already had high transaction volumes. The flow approach minimized latency and reduced the load on source systems because only changed data was transmitted. However, the team had to invest in robust monitoring and retry mechanisms to ensure that no events were lost, especially during system upgrades.

Lessons Learned

Key takeaway: Flow is essential when clinical safety depends on immediate data updates. However, it requires a mature infrastructure with proper error handling, monitoring, and capacity planning. The team also learned that idempotency is crucial to avoid duplicate medication lists when events are retried.

Scenario 3: Hybrid Approach for a Community Hospital

A community hospital with limited IT resources needed to integrate its EHR with a regional health information exchange (HIE), a lab system, and a state immunization registry. The EHR could support both scheduled and on-demand interfaces, but the team wanted to balance timeliness with operational simplicity. They adopted a hybrid approach: for lab results and immunization data, they used scheduled batches every four hours (thump) because the volume was moderate and clinicians could tolerate a few hours' delay for test results. For admission, discharge, and transfer (ADT) notifications to the HIE, they used on-demand flow because the HIE needed real-time bed availability data for care coordination.

The team built the thump interfaces using simple SQL queries and SFTP file transfers, which were easy to debug and monitor. The flow interface used HL7 over TCP/IP, with a dedicated queue that retried failed transmissions. The hybrid approach allowed the team to focus their limited expertise on the critical on-demand interface while keeping the lower-priority interfaces simple. They also created a weekly reconciliation job that cross-checked data between the EHR and HIE to catch any missed events.

The challenge they faced was ensuring that the scheduled batches did not conflict with the on-demand flows. For example, if a lab result was finalized shortly before a batch run, it would be included in the batch, but if the same result was also sent via a flow (if the lab system supported it), duplicates could occur. The team solved this by having the batch skip records that were already sent via flow within the last hour, using a timestamp comparison. This added some complexity but allowed them to benefit from both patterns.

Lessons Learned

Key takeaway: A hybrid approach can be pragmatic when resources are limited, but it requires careful coordination to avoid duplicates and ensure data consistency. The weekly reconciliation step was essential for maintaining trust in the data.

Common Mistakes and How to Avoid Them

Through our analysis of clinical integration projects, several recurring mistakes emerge. The first is assuming that on-demand flow is always superior. While low latency is desirable, the operational complexity of flow can outweigh the benefits for low-urgency data. Many teams jump to event-driven architectures without considering whether their infrastructure can support reliable, exactly-once event processing. The result is fragile integrations that lose events during peaks or require constant manual intervention.

The second mistake is polling as a default pattern. Polling (checking for changes every few minutes) seems like a compromise, but it combines the drawbacks of both thump and flow: it introduces latency (worst-case equal to poll interval) while also creating constant load on the source system. Polling is often a symptom of inadequate eventing capability in the source system, but it should be a last resort. Better alternatives include using database change data capture (CDC) triggers or a real-time API that supports subscriptions.

A third mistake is neglecting error handling in thump workflows. Because thump is simpler, teams sometimes assume that if a batch fails, they can simply rerun it. But rerunning a batch may cause duplicate records if the previous partial run already inserted some data. Proper idempotency handling (using upserts or unique keys) is necessary even in batch processes. Additionally, monitoring should detect not just batch failures but also batch successes that processed zero records—which could indicate a problem upstream.

Finally, many teams fail to consider the downstream impact of latency. Even if a thump batch runs every 15 minutes, the data in the consuming system is up to 15 minutes stale. For clinical decision support that relies on lab values, 15 minutes can be too long. Teams should formally document latency requirements with clinical stakeholders, not just assume what is acceptable.

Preventing Integration Debt

Integration debt accumulates when shortcuts are taken to meet deadlines. Common shortcuts include hardcoding schedules, skipping idempotency, and not documenting error handling procedures. To avoid this, treat integration design as a first-class engineering activity with the same rigor as application development. Conduct design reviews that specifically address the thump/flow decision.

Frequently Asked Questions

Q: Can I use both thump and flow for the same data feed? Yes, but you need to handle deduplication and conflict resolution. A common pattern is to use flow for immediate updates and a periodic thump for reconciliation to correct any missed events. This ensures both low latency and eventual consistency.

Q: Is on-demand always real-time? No. On-demand means triggered by an event, but the processing may involve queuing, transformation, and batching within the event pipeline. The end-to-end latency can still be seconds to minutes depending on the architecture. True real-time (sub-second) requires careful optimization.

Q: What is the impact of FHIR on the thump vs. flow decision? FHIR supports both: batch operations (e.g., $import, $export) for thump and subscriptions for flow. The choice should be driven by use case, not by the standard. FHIR subscriptions are a good option for flow when the source system supports them, but they require the subscriber to be reachable, which can be a challenge in network-segregated environments.

Q: How do I choose the batch interval for thump? The interval should be based on the maximum acceptable data staleness and the volume of data. A good starting point is to measure the peak change rate and set the interval so that the batch can complete within the time window before the next batch starts. Consider off-peak hours for large batches.

Q: What about security implications? Both patterns require secure transmission (TLS), but thump adds the challenge of storing credentials for scheduled jobs. Flow requires authentication for each event. In both cases, audit logging is critical. Thump may expose a larger attack surface if the batch window coincides with a vulnerability window.

Conclusion: Mastering the Rhythm of Clinical Integration

The conceptual workflow logic of scheduled vs. on-demand clinical integration is not about choosing one over the other universally, but about mapping the right rhythm to each use case. Thump offers predictability, simplicity in error recovery, and efficiency for high-volume transfers. Flow provides low latency, immediate responsiveness, and a natural fit for clinical workflows that demand timeliness. The most robust integration architectures use both, with clear boundaries and reconciliation mechanisms.

As you design or refine your clinical integrations, start by understanding the data's freshness requirements, the volume patterns, and the operational resilience of your systems. Use the decision framework provided here to systematically evaluate each interface. Avoid the common pitfalls of polling, neglecting error handling, and underestimating the complexity of event-driven systems. And remember that the goal is not to minimize latency at all costs, but to achieve the right balance of timeliness, reliability, and maintainability for your clinical context.

This guide is intended for general informational purposes only and does not constitute professional medical or legal advice. For specific integration decisions, consult with qualified healthcare IT professionals and regulatory experts.