August 20, 2026 – Taaja Tucker-Silva, Great Lakes Commission
While reviewing how the PAMF Model’s optimal and near-optimal guidance are calculated, we found that the portion of the model that generates near-optimal guidance was not behaving as expected. Luckily our optimal guidance was unaffected, but unfortunately, we found that the model was selecting near-optimal guidance recommendations that were essentially random. Clearly this is a problem! After consulting with the team that created the underlying R statistical package that the PAMF Model uses to generate its guidance, we updated the Model to rank and calculate near-optimal guidance as intended, finding those management combinations that are expected to perform within 5% as well at the optimal management combination.
What does this mean for you?
- Your 2026 guidance includes the new and improved near-optimal guidance!
- Your optimal guidance from past years has not changed
- Please disregard any near-optimal guidance from past years
Near-optimal guidance has improved substantially: when recalculating guidance from past years with the new model code, we found many fewer incidences of “Rest, Rest, Rest” as a recommended management combination (reductions of 70-94% per year!) resulting in more actionable combinations being recommended.
If you are interested in the mathematical details of the issue, read on! Otherwise, enjoy your new and improved management guidance!
Getting into the nitty gritty of the Model update
So, what exactly did we fix and why? First, there are some terms to understand:
Invasion state: The Phragmites invasion state of a management unit (MU), ranging from 1 to 6 (least to most Phragmites), is determined from PAMF monitoring reports. You can see your MU’s invasion state(s) in your Annual Management Unit Summary reports on the PAMF Web Hub.
Management restriction category: The management restriction category (bins 0-7, see table below) assigned to an MU is based on information from its enrollment report. Participants indicate whether herbicide treatment is allowed, whether Phragmites can be cut underwater, and whether water levels within the MU can be managed to enable flooding. Based on the combination of those answers, the MU is placed into a bin where only certain management combinations are evaluated:
Markov Decision Process (MDP): An MDP is a framework for making decisions when the outcome of each decision is uncertain and affects future options. This is the statistical structure underlying the PAMF Model and is implemented using the MDPToolbox package for R. There are four main parts to an MDP (there are others, but these are the important ones):
- State: Where you are now (the MU’s invasion state)
- Actions: What you can do (the management combinations allowed at an MU based on its restriction category)
- Transitions: What might happen after an action (the likelihood that an MU will transition from one invasion state to another between years).
- Rewards: The incentives that guide the decision-making process. In the PAMF Model, rewards include manager satisfaction and reduced cost of management.
Policy: In an MDP, a policy is simply a rule that tells the decision maker which action to take in each state. In the PAMF Model, these are our recommended management combinations (“guidance”).
When we run the PAMF model, for each invasion state and each restriction category, the PAMF MDP determines the management combinations that maximize rewards, and then produces optimal and near-optimal policies.
When we were evaluating the internal workings of the PAMF Model, we were looking at two measures (associated with the Bellman equation in the MDP) that had never been interrogated:
- State value V(s): the expected total reward of starting in a particular state s and then following a particular policy (e.g., a recommended management combination) all the way to the end.
- Action value Q(s,a): the expected total reward of taking a specific action a in a particular state s and then following a policy afterward for all subsequent steps.
V(s) measures your current standing, while Q(s,a) evaluates a choice, guiding your next move. In our model, V(s) is calculated for each of the optimal policies and compared to the Q(s,a) values of all the management combos to find near-optimal policies. When identifying near-optimal policies, max(Q(s,a)) from all possible management combinations should be ≈ V(s) of the optimal management combination.
When we did a deep dive into the model results, we noticed that max(Q(s,a)) was not ≈ V(s), and in fact was nowhere close. It turns out, after a conversation with the creator of the MDPToolbox package, that the function we were using to generate optimal policies correctly identifies the best policy, but does not return the best V(s) value. The V(s) estimates the function returned are approximations from the point where the algorithm stops running rather than the final optimal values.
To make sure we were using the most accurate V(s) estimates, we added an extra step in the Model code that calculates the final V(s) directly to match with max(Q(s,a)) produced by the function that calculates the near-optimal policies. We also reorganized the model output so that it now includes Q(s,a) values for every policy option, not just those classified as optimal or near optimal. This provides us with a complete view of how all policy options compare, and it allows us to quality check the Model’s decisions.
The full suite of data showing the differences between old-near-optimal guidance and new near-optimal guidance will be available in our next data release (see the section “PAMF Data & Information Dashboard” here). Here is an example of how near-optimal guidance has changed (looking at the data from the 2025 model re-run):
Our Model code updates are currently going through the USGS software release review process. All code updates can be seen in the PAMF Model code release here!