Modeling Patient Care Trajectories with Transformer Hawkes Processes

Patient healthcare utilization consists of irregularly time-stamped events, including outpatient visits, inpatient admissions, and emergency department encounters, which together form individualized care trajectories. Modeling such trajectories is essential for understanding utilization patterns and anticipating future care needs, but remains challenging due to temporal irregularity and severe class imbalance in real-world healthcare data. In this work, we build upon the Transformer Hawkes Process framework to model patient healthcare utilization trajectories in continuous time under realistic, highly imbalanced data conditions. By combining Transformer-based history encoding with a Hawkes point process formulation, the framework captures event-driven temporal dynamics and supports joint prediction of the next event type and the time gap until its occurrence. While prior Transformer-based point process models offer expressive temporal representations, they have not been systematically evaluated under the extreme event-type imbalance characteristic of healthcare utilization data. To address this limitation, we incorporate an imbalance-aware optimization strategy based on weighted cross-entropy, with class weights defined by the inverse square root of event frequencies. This approach preserves the natural temporal structure of patient trajectories while substantially improving sensitivity to rare but clinically significant inpatient and emergency events, without resorting to resampling or altering the underlying event distribution. We evaluate the resulting framework on real-world patient utilization data and demonstrate that this imbalance-aware extension enables more effective modeling of highly skewed care trajectories and improves both event-type prediction and time-to-event estimation. The resulting predictions provide clinically meaningful insights that can support healthcare systems in identifying, monitoring, and managing high-need, high-cost patient populations.