Research suggests that even small lapses from a weight control diet could explain suboptimal outcomes in behavioral weight loss programs. Just-in-time adaptive interventions (JITAI) delivered via smartphone apps can allow for the prediction and prevention of lapse behavior through momentary alerts to risk. Our team developed an app (OnTrack) that has been programmed to: (a) record data via ecological momentary assessment (EMA) on dietary lapse behavior and variables known to predict lapses, (b) apply machine learning to determine risk of a dietary lapse, (c) alert individuals when risk is high, and (d) offer a brief text-based micro-intervention (approximately 150-250 words) tailored to momentary risk factors. Phases I and II trials of OnTrack provided initial evidence for the feasibility, acceptability, and effectiveness of the app. However, during app development, there has been a tension between reducing participant burden by assessing fewer questions at each survey and having a more complete dataset from which to generate lapse predictions. A useful study of OnTrack could empirically examine the effect of additional EMA questions per prompt on app usage behaviors (i.e., EMA adherence, accessing the library of brief interventions, and opening alerts to lapse risk), user outcomes (i.e., lapses and weight change), and the quality of algorithm predictions. The study (n=121) evaluated these research questions by randomizing participants to use either the original app (i.e., 8 questions per prompt; OT-S) or an app with a longer question distribution scheme (i.e.,17 questions per prompt; OT-L) for ten weeks. Results indicate that OT-L and OT-S did not produce equivalent EMA compliance (MOT-s =65.35% v. MOT-L=60.49%; p=.16). Regardless, the completeness of cases did have a positive influence on the quality of algorithm predictions (MOT-S=50.70% v. MOT-L=57.9% accurate). Contrary to our hypotheses, improved predictive quality did not necessitate differences in accessing momentary interventions (MOT-s=47.66% v. M OT-L=46.15% risk alerts opened; p=.12) and this is likely due to problems that can arise while predicting an event that is also being intervened upon. Consequently, there were no differences in lapse frequency (MOT-s=3.3 v. MOT-L=3.1, p=.85) or weight change (MOT-s=-3.3 v. MOT-L=-3.5 PWL; p=.82) observed across conditions. Results from the current study have important implications for the next development phases of OnTrack, provide scientific insight into the relationship between app design and user behaviors, enhance our knowledge about the appropriate methods for using machine learning to personalize intervention, and add to existing literature regarding the effectiveness of just-in-time interventions for weight loss.