Herring Scrap 18

OR: “Courtroom Confusion About Models, Experts, and Accountability"

Last week, the Alaska Supreme Court issued an Opinion on Sitka Tribe of Alaska v. State of Alaska, Alaska Department of Fish & Game, and Southeast Herring Conservation Alliance (S18114).1 The opinion introduces the case with the words: “A tribe claimed that the State’s management of a commercial fishery harmed a subsistence fishery."

With this scrap, I hope to summarize the case and the opinion and my disappointment in the latest result. I’ll also sort of introduce ~~ duh duh duh ~~ the model. Barely; more in future scraps.

In brief:

The Alaska Supreme Court was considering Sitka Tribe of Alaska’s (STA) appeal of earlier rulings at the Superior Court level (for background, here is a KCAW story from last year). The original case had a few angles, but was generally about ADF&G failing to uphold a statutory, regulatory, and constitutional obligation to look out for the interests of subsistence users of herring and herring roe. In that case, Sitka Tribe prevailed on two of their key arguments (ADF&G was indeed failing statutory and regulatory obligations) but was denied on one (ADF&G was not failing the constitution). That constitutional piece is a big part of what the Alaska Supreme Court was reviewing for this appeal (question 1, below). Along the way, the Tribe was denied a preliminary injunction - the appropriateness of that denial was up for appeal before the Supreme court as well (question 2, below). The Tribe thought that they won enough in this case that attorney’s fees should be covered (question 3, below). Specifically, these are the questions that Sitka Tribe of Alaska brought to the Supreme Court on Jan 18, 2022 (in their Appellant Brief2, p1):

  1. Did the superior court err in concluding that the Sustained Yield Clause, Article VIII, section 4 of the Alaska Constitution, does not require the Alaska Department of Fish & Game (“ADF&G”) to provide the best available information to the Alaska Board of Fisheries?
  2. Did the superior court err in concluding that the injuries STA demonstrated to the subsistence way of life did not constitute irreparable harm?
  3. Did the superior court err in denying STA’s motion for attorney’s fees despite finding that STA had prevailed on the main issues of the litigation, which the court accurately characterized as going “to the very heart of this litigation”?

Yesterday the Alaska Supreme Court justices issued their opinion that the Superior Court did not err in any of those actions. I happen to disagree. So it goes.

I want to focus on the first question from above; the one about whether or not ADF&G (as a “Policy-implementing Agency”) has a constitutional obligation to provide the State of Alaska Board of Fisheries (as a “Policy-making Agency”) with “best available information” or “all relevant information” (the terms are used variously in the case materials).

To review — Superior Court said no, and then, last week, Alaska Supreme Court indicated their agreement: no. But how they got there gets more or less to the heart of why this whole thing about biomass is so deviously slippery and tediously important.

A Model Failure to Disclose

In making their case, STA had focused on a specific ADF&G failure to disclose key information to the Board of Fisheries: In 2018, ADF&G didn’t tell the Board of Fisheries that they were preparing to deploy a new model for the purposes of forecasting and hindcasting herring biomass in Sitka Sound.

This is what happened: ADF&G wanted to move on from the biomass forecast model they’d been using since 2014 and put out a call. A model-building contractor - Steve Martell - bid on a contract at the ADF&G biometrician’s invitation, won the contract, built the model, wrote documentation for it, and led a workshop it about it. All by 2016. The Board of Fisheries meetings were in early 2018; in the course of those meetings, ADF&G didn’t mention the new model that they were testing. The model would feature significant changes to how the herring biomass is forecast and historicized. Meanwhile: up for deliberation were a pair of proposals to change the Guideline Harvest Level from 10-20% to 0-10% of biomass.

And here’s the thing: by all appearances this new model is a booster pack for biomass. For most years, it seems that the model crunches the same data and spits out a bigger biomass (see Fig 2 of Sullivan 2018, below)3 than the prior model would have.

This is Figure 2 from Sullivan 2018, comparing the previous model (labeled LS, dashed grey line) with the new model (labeled HER, solid black line). Original caption text reads: “Model-estimated mature biomass in HER (black solid line) and LS (grey dashed line) compared to historical estimates from the survey (open circles) that convert survey egg deposition to biomass using catch, age composition, and weight-at-age data. […]”

Given that how the biomass is measured has a direct bearing on the Guideline Harvest Level of the commercial fishery, I really think that Sitka Tribe of Alaska has a point here. It should have been said in those meetings - it was highly relevant - that this new model came with different assumptions and different biases than the ones before and that it would have something to do with the GHL, that it might even have a tendency to result in higher biomass estimates than the previous model, and that in some years it would bring a not-insignificant boost to the GHL4.

Put another way: The model is a storytelling risk assessment machine. It is using all of this information from all of these sources to tell a story about biomass. One such machine was replaced by another. The two tell different stories5. But those stories, and the differences, weren’t articulated for the public or for the Board of Fisheries.

I watched as dozens of people spoke up for those proposals, while the Department sat on the knowledge that they were about to uncork a new model that might artificially boost the GHL by hundreds and thousands of tons ever year. Stinks!

But Alaska Supreme Court says: no problem. ADF&G does not have a constitutional obligation to provide the Board of Fisheries with all relevant information.

Expertise

The decision came down to ideas about expertise.

Take a look at this excerpt from the Opinion (p14) where the model and it’s author are (inaccurately) described:

I don’t mean to pick on the justices here, but I also think it’s important to point out: they do not get it right when it comes to what the report IS (the documentation for a biomass model) and what the role of the author is (he was hired to build a model). It’s not crystal clear that the justices understand the “report” is actually the documentation for an existing model… and that the author of the “report” had already built the model, it had been tested by the agency and was near-ready for implementation at the time of the 2018 Board of Fisheries meeting.

One of the tensions that came up at the Superior Court in Juneau and then again at the Alaska Supreme Court was that the justices were wary of overstepping into spaces of privileged expertise and arcane knowledge, expressing reluctance to cast judgement around the ins and outs of (for example) the care and feeding of biomass models. And that they weren’t able to successfully describe the model or it’s role or the author’s role in its creation is almost exactly the point: these justices have no business making heads or tails of such things. I get it. And in their Opinion, they extend the same grace to the members of the Board of Fisheries, writing:

“The hard look standard already requires the Department to consider relevant information and ‘engage[] in reasoned decision making.’ The Department’s decision to not provide the report to the Board was not arbitrary because it was a highly technical report mostly concerned with computer coding fixes to the biomass forecasting program. We therefore decline to create a constitutional requirement[…]”

We’ve seen enough to know that this is nerd stuff, they nearly said, and thus it is fine and reasonable for decision makers to be insulated from it. And that’s why they ruled against Sitka Tribe of Alaska on this one.

And so it is that by its arcane nature, the model and the biomass that it spits out have a special legal status. Everybody just has to agree: it’s super magical, as good as true, a paragon of neutrality. To even have a contrary opinion about it, you must be an expert at it. To be an expert at it, you must work at ADF&G. And I plea, as they drag me away: a restructured model is a big deal! This isn’t about debugging something. This is about re-designing a construct for harvest allocation. If decision makers aren’t expected to talk about it, we might as well hand fisheries management over to AI. (please no).

Experts

In closing here, I’m reminded by all of this, as I often am, of Mark Jacobs Jr.’s cautionary comments to the Board of Fisheries in 1997:

“I've lost my confidence in all this expertise, so-called: Knowledgable. Scientific. Biologists.”

and later, having been asked no questions following his testimony,

“I wish you had cross-examined me on the expert biologists.
They can tell you a lot
But they are not experts.”

**

I’ll leave things there for now. Thanks for reading! Do write.

pb


  1. Winfree, Chief Justice, Carney and Henderson, Justices, Fabe and Bolger, Senior Justices. (2023). OPINION on SITKA TRIBE OF ALASKA v STATE OF ALASKA, DEPARTMENT OF FISH AND GAME, and SOUTHEAST HERRING CONSERVATION ALLIANCE, (Supreme Court No. S-18114      Trial Court Case No 1SI-18-00212CI).

  2. Brief of Appellant Sitka Tribe of Alaska. SITKA TRIBE OF ALASKA v STATE OF ALASKA, DEPARTMENT OF FISH AND GAME, and SOUTHEAST HERRING CONSERVATION ALLIANCE (Supreme Court No. S-18114  Trial Court Case No 1SI-18-00212CI). Filed Jan 18, 2022.

  3. Anyhow, on their github page, the Department has a stash of materials from the model transition, including Sullivan 2018, a comparison of the 2018 biomass forecast using the old model (LS) alongside the new model (HER). The new model provided the fleet with a GHL 2,362 tons higher than the previous model. The author of the comparison wrote: “The LS forecast of mature biomass was 55,637 tons and the corresponding guideline harvest level (GHL) was 11,127 tons (20% harvest rate). The HER forecast of mature biomass was 61,187.81, resulting in a GHL of 13,489.67.”

  4. For those wondering: the model is fed egg deposition estimates and age composition surveys,

  5. Here’s one example, from those same project notes on github (Sullivan 2018): “The structure and estimation of natural mortality is very different between models.”