Artificial intelligence or machine learning techniques are currently being widely applied for solving problems within the field of data analytics. This work presents and demonstrates the use of a new machine learning algorithm for solving semi-Markov decision processes (SMDPs). SMDPs are encountered in the domain of Reinforcement Learning to solve control problems in discrete-event systems. The new algorithm developed here is called iSMART, an acronym for imaging Semi-Markov Average Reward Technique. The algorithm uses a constant exploration rate, unlike its precursor R-SMART, which required exploration decay. The major difference between R-SMART and iSMART is that the latter uses, in addition to the regular iterates of R-SMART, a set of so-called imaging iterates, which form an image of the regular iterates and allow iSMART to avoid exploration decay. The new algorithm is tested extensively on small-scale SMDPs and on large-scale problems from the domain of Total Productive Maintenance (TPM). The algorithm shows encouraging performance on all the cases studied.