We consider the single-server Markovian queue with infinite waiting space and assume that there exists a certain reward-cost structure that reflects the customers’ desire for service and their dislike for waiting. The system is unobservable for the customers at their arrival instants, but the administrator provides them with periodic announcements of their current positions at rate (Formula presented.), so that they may renege if it is preferable for them to do so. The customers are strategic, and their decision problem is whether to join or not the system upon arrival and whether to stay or renege later. Their strategies are specified by a join probability q and a reneging threshold n. We determine the equilibrium strategies (Formula presented.) and study the socially optimal strategies (Formula presented.). Extensive numerical experiments provide interesting qualitative insight about the model. In particular, the equilibrium throughput of the system is a unimodal function of (Formula presented.). Moreover, despite the fact that we have an avoid-the-crowd situation, it is possible that (Formula presented.), in contrast to the classical unobservable model. © 2017 Springer Science+Business Media New York

We consider the problem of customer equilibrium strategies in an M/M/1 queue under dynamic service control. The service rate switches between a low and a high value depending on system congestion. Arriving customers do not observe the system state at the moment of arrival. We show that due to service rate variation, the customer equilibrium strategy is not generally unique, and derive an upper bound on the number of possible equilibria. For the problem of social welfare optimization, we numerically analyze the relationship between the optimal and equilibrium arrival rates as a function of various parameter values, and assess the level of inefficiency via the price of anarchy measure. We finally derive analytic solutions for the special case where the service rate switch occurs when the queue ceases to be empty. © 2016 Elsevier B.V. All rights reserved.

We consider the multi-armed bandit problem under a cost constraint. Successive samples from each population are i.i.d. with unknown distribution and each sample incurs a known population-dependent cost. The objective is to design an adaptive sampling policy to maximize the expected sum of n samples such that the average cost does not exceed a given bound sample-path wise. We establish an asymptotic lower bound for the regret of feasible uniformly fast convergent policies, and construct a class of policies, which achieve the bound. We also provide their explicit form under Normal distributions with unknown means and known variances. Copyright © Cambridge University Press 2016

We consider a two node supply chain with a rational manufacturer-retailer pair, in which the retailer has private information that affects the nodes[U+05F3] reservation levels. Quantity discounts offered by the manufacturer is the mechanism we propose in order to achieve reduced costs for both supply chain nodes. We derive analytical expressions of the quantity discounts that minimize the manufacturer[U+05F3]s costs, while enabling the establishment of the business. Furthermore, we show that perfect coordination is possible even under asymmetric information. Sensitivity analysis and numerical examples offer evidence of the robustness of the results and of the potential of the approach for applications to real-life business ventures. © 2014 Elsevier Ltd.

This article studies the dynamic control of arrivals of multiple job classes in N-stage production systems with finite buffers and blocking after service. A model with multiple processing stages in series is formulated as a Markov decision process and a state definition from the queueing analysis literature is used to simplify the state-space description. This allows several fundamental admission control results from M/M/N and M/M/N/N queueing models as well as tandem models without blocking to be extended to tandem systems with blocking. Specifically, it is shown that the net benefit of admitting a job declines monotonically with the system congestion; thus the decision to admit any job class is based on threshold values of the number of jobs present in the system. Furthermore, conditions under which a job class is always or never admitted, regardless of the state, are derived. The interaction of blocking and admission control is explored by analyzing the effect of blocking on the optimal admission policy and profit. The article concludes with analyses of why extensions including loss and abandonment cannot sustain the monotonicity properties and two surrogate admission rules that may be used in practice but do not account for the blocking effect. © 2013 Taylor & Francis Group, LLC.

We consider series of M/M/m queues with strategic customer behavior. Customers arrive to the first queue and decide whether to enter the system or balk and, if they enter, up to which queue to proceed before departing. Each customer makes an independent decision, with the objective of maximizing her total net benefit, which is equal to the value of service minus a cost due to expected delay. We formulate the customer decision as a game and identify the unique symmetric Nash equilibrium strategy, which is expressed in a backward recursive form. We also analyze the problem of maximizing the total customer welfare and establish the relationship between the equilibrium and the welfare maximizing strategies. © 2011 Springer Science+Business Media, LLC.

The propagation stage of uncertainty evaluation, known as the propagation of distributions, is in most cases approached by the GUM (Guide to the Expression of Uncertainty in Measurement) uncertainty framework which is based on the law of propagation of uncertainty assigned to various input quantities and the characterization of the measurand (output quantity) by a Gaussian or a t-distribution. Recently, a Supplement to the ISO-GUM was prepared by the JCGM (Joint Committee for Guides in Metrology). This Guide gives guidance on propagating probability distributions assigned to various input quantities through a numerical simulation (Monte Carlo Method) and determining a probability distribution for the measurand. In the present work the two approaches were used to estimate the uncertainty of the direct determination of cadmium in water by graphite furnace atomic absorption spectrometry (GFAAS). The expanded uncertainty results (at 95% confidence levels) obtained with the GUM Uncertainty Framework and the Monte Carlo Method at the concentration level of 3.01 μg/L were ±0.20 μg/L and ±0.18 μg/L, respectively. Thus, the GUM Uncertainty Framework slightly overestimates the overall uncertainty by 10%. Even after taking into account additional sources of uncertainty that the GUM Uncertainty Framework considers as negligible, the Monte Carlo gives again the same uncertainty result (±0.18 μg/L). The main source of this difference is the approximation used by the GUM Uncertainty Framework in estimating the standard uncertainty of the calibration curve produced by least squares regression. Although the GUM Uncertainty Framework proves to be adequate in this particular case, generally the Monte Carlo Method has features that avoid the assumptions and the limitations of the GUM Uncertainty Framework. © 2010 Elsevier B.V. All rights reserved.

This paper considers the problem of optimal pricing in a system serving two classes of customers differentiated by their delay sensitivities. We derive the revenue maximising pricing policies whether or not price discrimination is an option. We find that in both cases the optimal policy causes the less delaysensitive class to enter first, and the optimal prices are increasing in capacity under price discrimination, which is not generally true when price discrimination is not allowed. Furthermore, under price discrimination, less capacity is needed to capture a customer class or the entire market, while, more customers are served and higher revenue is generated. Finally, we use an M/M/1 system to provide further insights and numerical analysis. © 2011 Inderscience Enterprises Ltd.

We consider a Paris Metro Pricing (PMP) approach for providing service to two classes of customers differentiated by their delay sensitivity. We develop a leader-follower game, where the leader is the service provider who sets the price and the customers respond by deciding whether to join or balk. We derive the customer behaviour as the Nash equilibrium of a multi-person game and obtain the revenue maximising price pairs for all combinations of arrival rates from each class to each server. We finally derive the capacity threshold in such domain and its impact on customer accessibility to the product or service. Copyright © 2009, Inderscience Publishers.

We study a system where the service provider offers priority options. We identify the optimal option pricing policy, by deriving the optimal number a customer would buy and the customer's exercise policy as a function of system congestion, options remaining, time to expiration and possibility of balking.

We study the effects of disruption risk in a supply chain where one retailer deals with competing risky suppliers who may default during their production lead times. The suppliers, who compete for business with the retailer by setting wholesale prices, are leaders in a Stackelberg game with the retailer. The retailer, facing uncertain future demand, chooses order quantities while weighing the benefits of procuring from the cheapest supplier against the advantages of order diversification. For the model with two suppliers, we show that low supplier default correlations dampen competition among the suppliers, increasing the equilibrium wholesale prices. Therefore the retailer prefers suppliers with highly correlated default events, despite the loss of diversification benefits. In contrast, the suppliers and the channel prefer defaults that are negatively correlated. However, as the number of suppliers increases, our model predicts that the retailer may be able to take advantage of both competition and diversification. © 2007 INFORMS.

We consider a single server Markovian queue with setup times. Whenever this system becomes empty, the server is turned off. Whenever a customer arrives to an empty system, the server begins an exponential setup time to start service again. We assume that arriving customers decide whether to enter the system or balk based on a natural reward-cost structure, which incorporates their desire for service as well as their unwillingness to wait. We examine customer behavior under various levels of information regarding the system state. Specifically, before making the decision, a customer may or may not know the state of the server and/or the number of present customers. We derive equilibrium strategies for the customers under the various levels of information and analyze the stationary behavior of the system under these strategies. We also illustrate further effects of the information level on the equilibrium behavior via numerical experiments. © 2007 Springer Science+Business Media, LLC.

We investigate how a supplier can use a quantity discount schedule to influence the stocking decisions of a downstream buyer that faces a single period of stochastic demand. In contrast to much of the work that has been done on single-period supply contracts, we assume that there are no interactions between the supplier and the buyer after demand information is revealed and that the buyer has better information about the distribution of demand than does the supplier. We characterize the structure of the optimal discount schedule for both all-unit and incremental discounts and show that the supplier can earn larger profits with an all-unit discount.

We consider the problem of dynamic admission control in a Markovian loss system with two classes. Jobs arrive at the system in batches; each admitted job requires different service rates and brings different revenues depending on its class. We introduce the definition of a 'preferred class' for systems receiving mixed and single-class batches separately, and derive sufficient conditions for each system to have a preferred class. We also establish a monotonicity property of the optimal value functions, which reduces the number of possibly optimal actions. © Applied Probability Trust 2005.

Mathematical models are presented for studying the value of leadership in a team where the members interact with each other. The models are based on a leader's role of motivating each team member to perform closer to his/her maximum ability. These models include controllable parameters whose values reflect the amount of task interdependence among the workers as well as the motivational skill and variability in the skill of the leader. Confirming results - such as the fact that the skill level of the leader is a critical factor in the expected performance of the team - establish credibility in the models. Mathematical analysis and computer simulations are used to provide new managerial insights into the value of the leader - such as the fact that the skill of the leader can be more important than controlling the amount of interdependence among the team members and that having a choice of multiple leaders with no particular motivating skill is beneficial to the performance of small teams but not to large teams. © Springer Science + Business Media, Inc. 2005.

This article investigates the role of option contracts in a supply chain when the demand curve is downward sloping. We consider call (put) options that provide the retailer with the right to reorder (return) goods at a fixed price. We show that the introduction of option contracts causes the wholesale price to increase and the volatility of the retail price to decrease. In general, options are not zero-sum games. Conditions are derived under which the manufacturer prefers to use options. When this happens the retailer is also better off, if the uncertainty in the demand curve is low. However, if the uncertainty is sufficiently high, then the introduction of option contracts alters the equilibrium prices in a way that hurts the retailer. © 2005 INFORMS.

We consider the problem of dynamic admission control in a multi-class Markovian loss system receiving random batches, where each admitted class-i job demands an exponential service with rate μ, and brings a reward r i. We show that the optimal admission policy is a sequential threshold policy with monotone thresholds. © 2004 Elsevier B.V. All rights reserved.

The multiarmed-bandit problem is often taken as a basic model for the trade-off between the exploration and utilization required for efficient optimization under uncertainty. In this article, we study the situation in which the unknown performance of a new bandit is to be evaluated and compared with that of a known one over a finite horizon. We assume that the bandits represent random variables with distributions from the one-parameter exponential family. When the objective is to maximize the Bayes expected sum of outcomes over a finite horizon, it is shown that optimal policies tend to simple limits when the length of the horizon is large.

We consider the problem of dynamic admission control in a Markovian loss queueing system with two classes of jobs with different service rates and random revenues. We establish the existence of an optimal monotone policy. We also show that under certain conditions there exist preferred jobs from either class.

In this paper we are going to develop group replacement policies for a service system by considering the number of customers dynamically in the queue. We consider a service system with multiple independent servers operating in parallel and a single queue. Customers arrive in accordance with a Poisson process, and the service time for each customer follows an exponential distribution. The servers are unreliable with identically exponentially distributed failure times and the repair time is assumed negligible. We formulate this model as a continuous time Markov decision process, and prove the optimal group replacement policy has a threshold structure.

We consider the problem of dynamic admission control in a Markovian loss queueing system with two classes of customers with different service rates and revenues. We show that under certain conditions, customers of one class, which we call a preferred class, are always admitted to the system. Moreover, the optimal policy is of threshold type, and we establish that the thresholds are monotone under very restrictive conditions. Copyright © 2001 by Marcel Dekker, Inc.

In this paper we study a situation in which a broker must manage the procurement of a short-life-cycle product. As the broker observes demand for the item, she learns about the demand process. However, as is often the case in practice, it becomes either more difficult or more expensive to procure the item as the selling season advances. Thus, the broker must trade off higher procurement costs against the benefit of making ordering decisions with better information about demand. Problems of this type arise, for example, in the travel industry, where a travel agent's cost of procuring airline and hotel reservations increases as the date of a vacation package approaches. We develop a newsvendor-like characterization of the optimal procurement policy. In a numerical analysis, we demonstrate how broker procurements tend to cluster just before price increases and how brokers can benefit from explicitly considering the effects of information about demand in their ordering policies.

We consider the combined problem of pricing and ordering for a perishable product with unknown demand distribution and censored demand observations resulting from lost sales, faced by a monopolistic retailer. We develop an adaptive pricing and ordering policy with the asymptotic property that the average realized profit per period converges with probability one to the optimal value under complete information on the distribution. The pricing mechanism is modeled as a multiarmed bandit problem, while the order quantity decision, made after the price level is established, is based on a stochastic approximation procedure with multiplicative updates.

Mathematical analysis and computer simulations are used to evaluate three modifications to Kauffman's NK model in an attempt to incorporate unexplored aspects of epistatic interaction between loci in genome evolution. Two modifications - one to the amount and the other to the distribution of epistatic interaction - further support Kauffman's conclusion that high levels of epistatic interaction lead to a decrease in overall fitness of the genome. The third model, however, provides a condition under which increased epistatic interaction at certain loci results in higher genome fitness.

The unknown performance of a new experiment is to be evaluated and compared with that of an existing one over a finite horizon. The explicit structure of an optimal sequential allocation policy is obtained under pertinent reward/loss functions, when the experiments are characterized by random variables with distributions from the one parameter exponential family.

Various methods, such as address-calculation sorts, distribution counting sorts, radix sorts, and bucket sorts, use the values of the numbers being sorted to increase efficiency but do so at the expense of requiring additional storage space. In this paper, a specific implementation of bucket sort is presented whose primary advantanges are that (i) linear average-time performance is achieved with an additional amount of storage equal to any fraction of the number of elements being sorted and (ii) no linked-list data structures are used (all sorting is done with arrays). Analytical and empirical results show the trade-off between the additional storage space used and the improved computational efficiency obtained. Computer simulations show that for lists containing 1,000 to 30,000 uniformly distributed positive integers, the sort developed here is faster than both Quicksort and a standard implementation of bucket sort. Furthermore, the running time increases with size at a slower rate.

Consider a finite state irreducible Markov reward chain. It is shown that there exist simulation estimates and confidence intervals for the expected first passage times and rewards as well as the expected average reward, with 100% coverage probability. The length of the confidence intervals converges to zero with probability one as the sample size increases; it also satisfies a large deviations property.

In this paper we consider the problem of adaptive control for Markov Decision Processes. We give the explicit form for a class of adaptive policies that possess optimal increase rate properties for the total expected finite horizon reward, under sufficient assumptions of finite state-action spaces and irreducibility of the transition law. A main feature of the proposed policies is that the choice of actions, at each state and time period, is based on indices that are inflations of the right-hand side of the estimated average reward optimality equations.

Consider the problem of sequential sampling from m statistical populations to maximize the expected sum of outcomes in the long run. Under suitable assumptions on the unknown parameters θ= ∈ Θ, it is shown that there exists a class CR of adaptive policies with the following properties: (i) The expected n horizon reward Vπ0 n (θ=) under any policy π0 in CR is equal to nμ,*(η=) - M(θ=)log n + o(log n), as n → ∞, where μ*(θ=) is the largest population mean and M(θ=) is a constant. (ii) Policies in CR are asymptotically optimal within a larger class CUF of "uniformly fast convergent" policies in the sense that limn → ∞(nμ,*(θ=) - Vπ0 n (θ=))/ (nμ*(θ=) - Vπ n(θ=)) ≤ 1, for any π ∈ CUF and any θ= ∈ Θ such that M(θ=) > 0. Policies in CR are specified via easily computable indices, defined as unique solutions to dual problems that arise naturally from the functional form of M(θ=). In addition, the assumptions are verified for populations specified by nonparametric discrete univariate distributions with finite support. In the case of normal populations with unknown means and variances, we leave as an open problem the verification of one assumption. © 1996 Academic Press, Inc.

A simulation method is developed for computing average reward optimal policies, for a finite state and action Markovian decision process. It is shown that the method is consistent; i.e., it produces solutions arbitrarily close to the optimal. Various types of estimation errors and confidence bounds are examined. Finally, it is shown that the probability distribution of the number of simulation cycles required to compute an é-optimal policy satisfies a large deviations property. © 1995, Cambridge University Press. All rights reserved.

We consider the problem of sequential control for a finite state and action Markovian Decision Process with incomplete information regarding the transition probabilities P ∈ Papprox. Under suitable irreducibility assumptions for Papprox.. We construct adaptive policies that maximize the rate of convergence of realized rewards to that of the optimal (non adaptive) policy under complete information. These adaptive policies are specified via an easily computable index function, of states, controls and statistics, so that one takes a control with the largest index value in the current state in every period.

Two types of tasks are to be scheduled on a single processor under incomplete information about the task lengths. We derive the structure of optimal scheduling rules w.r.t. flowtime, as well as asymptotic approximations for a large number of tasks, when the length distributions belong to a one-parameter exponential family. © 1993, Cambridge University Press. All rights reserved.