本研究旨在探討連鎖企業在需求不確定性下，如何有效地分配有限的運輸資源和決定庫存水平，以最小化總庫存成本。當一家具有多家連鎖店的連鎖企業受到運輸資源的限制，無法在每一期滿足所有連鎖店的補貨需求時，便需要在所有店家中做出補貨與否的選擇。同時，由於需求未知的緣故，決策者必須透過販售商品的方式來學習需求，並以此評估與決定連鎖店的庫存水平。為了提供決策者選擇補貨商家以及決定庫存水平的兩項有效決策，本研究結合庫存模型與多臂拉霸機(multi-armed bandit)模型，並且在考慮需求學習與審查(censor)的情況下，提出一項近似Whittle index演算法進行求解，以此決定資源分配的優先順序。本研究進一步改進了Whittle index演算法，以針對本研究所關注的補貨問題進行求解。利用此演算法，可以計算出每家連鎖店在不同的狀態下的優先指標，再透過比較各家連鎖店的指標大小，提供決策者有效且直覺的選擇補貨店家之方法。在建立完模型與演算法後，本研究也針對不同的情況進行數值分析。透過調整各個參數的高低，將此演算法與常見的短視近利(myopic)策略、信賴上界(UCB)策略以及上帝視角(oracle)進行比較。最終，我們的研究結果明確顯示近似Whittle index演算法在有限運輸資源下，連鎖企業在庫存管理方面的重要價值。該演算法確實能夠有效地優化資源分配，尤其當決策者面臨多樣化的選擇補貨店家的情境時，將展現出更快速收斂至上帝視角的優勢，為庫存管理和資源分配帶來實質的增益。

This study aims to investigate how chain enterprises can effectively allocate limited transportation resources and determine inventory levels to minimize total inventory costs under demand uncertainty. When a chain enterprise with multiple stores faces constraints on transportation resources and cannot fulfill the replenishment demands of all stores in each period, decisions need to be made regarding which stores to replenish. Additionally, due to the unknown demand, decision-makers must learn the demand through product sales and use this information to evaluate and determine the inventory levels for each store. To provide decision-makers with effective strategies for selecting replenishment stores and determining inventory levels, this study combines inventory models with multi-armed bandit models and proposes an approximate Whittle index algorithm for solution, considering demand learning and censoring. The improved algorithm calculates the priority indexes for each store in different states and facilitates decision-making by comparing these indexes. Numerical analyses are conducted by comparing the proposed algorithm with common myopic, upper confidence bound (UCB), and oracle strategies under various scenarios. The results clearly demonstrate the significant value of the approximate Whittle index algorithm in inventory management for chain enterprises with limited transportation resources. The algorithm effectively optimizes resource allocation, particularly in situations involving diverse choices of replenishment stores, exhibiting faster convergence towards an oracle perspective and providing substantial benefits to inventory management and resource allocation.

Abbou, A., Makis, V., & Salari, N. (2017). An index policy for scheduling group maintenance services. In IIE Annual Conference. Proceedings, (pp. 295–300). Institute of Industrial and Systems Engineers (IISE).

Araman, V. F., & Caldentey, R. (2009). Dynamic pricing for nonperishable products with demand learning. Operations research, 57(5), 1169–1188.

Ayer, T., Zhang, C., Bonifonte, A., Spaulding, A. C., & Chhatwal, J. (2019). Prioritizing hepatitis C treatment in U.S. prisons. Operations Research, 67(3), 853–873.

Battini, D., Persona, A., & Sgarbossa, F. (2014). A sustainable EOQ model: Theoretical formulation and applications. International Journal of Production Economics, 149, 145–153.

Bensoussan, A., Sethi, S., Thiam, A., & Turi, J. (2021). Optimality of base stock policy under unknown demand distributions: New results and a new method. Available at SSRN 3858415.

Besbes, O., Chaneton, J. M., & Moallemi, C. C. (2022). The exploration-exploitation trade-off in the newsvendor problem. Stochastic Systems.

Braden, D. J., & Freimer, M. (1991). Informational dynamics of censored observations. Management Science, 37(11), 1390–1404.

Bu, J., Gong, X., & Chao, X. (2022). Asymptotic optimality of base-stock policies for perishable inventory systems. Management Science.

Caro, F., & Gallien, J. (2010). Inventory management of a fast-fashion retail network. Operations Research, 58(2), 257–273.

Chen, L. (2010). Bounds and heuristics for optimal bayesian inventory control with unobserved lost sales. Operations Research, 58(2), 396–413.

Davis, A. M., Huang, R., & Thomas, D. J. (2022). Retailer inventory sharing in two-tier supply chains: An experimental investigation. Management Science, 68(12), 8773– 8790.

Farias, V. F., & Gutin, E. (2022). Optimistic Gittins indices. Operations Research, 70(6), 3432–3456.

Fisher, M. (2009). Rocket science retailing: The 2006 Philip McCord Morse lecture. Operations Research, 57(3), 527–540.

Gao, X., Jasin, S., Najafi, S., & Zhang, H. (2022). Joint learning and optimization for multi-product pricing (and ranking) under a general cascade click model. Management Science, 68(10), 7362–7382.

Gittins, J. C. (1979). Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society: Series B (Methodological), 41(2), 148–164.

Glazebrook, K. D., & Minty, R. (2009). A generalized Gittins index for a class of multiarmed bandits with general resource requirements. Mathematics of Operations Research, 34(1), 26–44.

Graczova, D., & Jacko, P. (2014). Generalized restless bandits and the knapsack problem ´ for perishable inventories. Operations Research, 62(3), 696–711.

Huh, W. T., & Rusmevichientong, P. (2009). A nonparametric asymptotic analysis of inventory planning with censored demand. Mathematics of Operations Research, 34(1), 103–123.

Kawakami, K., Kobayashi, H., & Nakata, K. (2021). Seasonal inventory management model for raw materials in steel industry. INFORMS Journal on Applied Analytics, 51(4), 312–324.

Keskin, N. B., & Zeevi, A. (2017). Chasing demand: Learning and earning in a changing environment. Mathematics of Operations Research, 42(2), 277–307.

Lariviere, M. A., & Porteus, E. L. (1999). Stalking information: Bayesian inventory management with unobserved lost sales. Management Science, 45(3), 346–363.

Li, R., Song, J.-S. J., Sun, S., & Zheng, X. (2022). Fight inventory shrinkage: Simultaneous learning of inventory level and shrinkage rate. Production and Operations Management, 31(6), 2477–2491.

Mate, A., Biswas, A., Siebenbrunner, C., & Tambe, M. (2021). Efficient algorithms for finite horizon and streaming restless multi-armed bandit problems. arXiv preprint arXiv:2103.04730.

Nahmias, S. (1994). Demand estimation in lost sales inventory systems. Naval Research Logistics (NRL), 41(6), 739–757.

Nambiar, M., Simchi-Levi, D., & Wang, H. (2021). Dynamic inventory allocation with demand learning for seasonal goods. Production and Operations Management, 30(3), 750–765.

Puterman, M. L. (2014). Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons.

Russo, D. (2021). A note on the equivalence of upper confidence bounds and gittins indices for patient agents. Operations Research, 69(1), 273–278.

Scarf, H. (1959). Bayes solutions of the statistical inventory problem. The Annals of Mathematical Statistics, 30(2), 490–508.

Song, J.-S., & Xue, Z. (2021). Demand shaping through bundling and product configuration: A dynamic multiproduct inventory-pricing model. Operations Research, 69(2), 525–544.

Vapnik, V. N. (1999). An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5), 988–999.

Whittle, P. (1988). Restless bandits: Activity allocation in a changing world. Journal of Applied Probability, 25(A), 287–298.

Zhang, H., Chao, X., & Shi, C. (2018). Perishable inventory systems: Convexity results for base-stock policies and learning algorithms under censored demand. Operations Research, 66(5), 1276–1286.