Demographics

When looking at all of this data, remember that we’re only looking at a small subset of people that (a) I reached and (b) responded. Of course, there’s quite a bit of sample bias here.

• 97 responses, mostly from Europe (43) and North America (30).
• Most respondents are applied scientists (62) or coders and data scientists (26).
• Nearly a third (29) of respondents are in organizations with more than 10 000 employees; about a third (34) in orgs smaller than 100.
• Respondents work in service and technology companies (30), operating companies (27) and academia (21):

Open source

To try to capture the current state, most of the questions in all sections asked respondents to consider only the most recent six months when answering.

• 65 respondents work in teams that strongly prefer or lean towards open-source technical tooling.
• 67 respondents use open-source software at least weekly for subsurface workflows.
• When people choose open options for technical work, three reasons dominate: extensiblity, transparency (of functionality like algorithms) and cost, with 56 to 60 respondents choosing each of these features.

Maintaining projects

• A third of respondents (33) are maintainers (this role includes core contributors) of open-source projects. Of these, 27 help maintain at least one subsurface-related project.
• 12 of the 33 maintainers work alone on their primary project; 4 of 33 have six or more co-maintainers.
• 9 of the primary projects of the 33 maintainers are unfunded.
• I asked how maintainers feel about some specific tasks in the near future:

Contributing to projects

• 39 respondents contributed to at least one project they do not maintain. A few (7) people make at least one contribution per month.
• The main barrier to contribution is a lack of time, incentive, or permission (perceived or otherwise), with 47 respondents choosing these reasons.
• The top contribution types: creating issues (25 respondents, so 64% of those making contributions), small documentation fixes (18, or 46%), non-critical bug fixes (16), new tests (14), and new features (13).

Artificial intelligence

Open source has a complicated relationship with AI, as most of the rest of the world. Maintainers and would-be contributors sense it could help with some tasks, while it also seems to present some new security challenges — and the threat of sloppy contributions that no-one wants to review.

• 60 respondents use AI assistants or agentic tools daily; 12 rarely or never use them.
• Those using AI less often tend to have less trust in the technology (see below).
• There is no correspondence between trust and those having maintainer roles:

• Respondents are slightly more concerned about IP issues in upstream dependencies (only 22 not concerned) than they are about introducing such problems into their own code (28 not concerned).
• People think AI may improve access to technology: 71 respondents agree that AI code assitants lower the barrier for non-programmers to use digital products, while 51 respondents agree that AI lowers the barrier for non-programmers to contribute to projects.

Organizations

• Only 16 respondents report leader resistance to open source; 59 respondents report support or enthusiasm among leadership.
• Only 7 respondents work in organizations that have eastablished open source program offices (OSPOs). Only 12 report their organization funding open source activities.
• 31 have organizations with expressed preference for open source, with 20 respondents reporting a “clear and visible” open source strategy.
• Some respondents have been affected by various kinds of disruption to open source projects they depend on:

The future

The last question was about the health of the subsurface open source ecosystem. While only 3 people characterize the situation as ‘critical’, 40 respondents (41%) believe it to be at risk. On the other hand, 57 think it is at least stable and a handful believe it to be ‘robust’.

An approximately normal distribution, but skewed positive — a good description of the community itself!

All the data and results are on GitHub

The fourth edition

This was the fourth EAGE open source workshop. Here’s the timeline:

• 2006 — EAGE, Vienna
• 2012 — EAGE, Copenhagen — blog post
• 2016 — EAGE, Vienna — blog post
• 2026 — EAGE, Aberdeen — blog post

To our knowledge, there have been two other events in a similar vein: one PTTC meeting, and an Agile event that was effectively the Software Underground’s first conference:

• 2011 — PTTC, Houston — blog post
• 2019 — TRANSFORM #1, Rouen — blog post

Coding agents as users? Contributors? Maintainers?

One of the patterns I (Matt) learned from Leo Uieda and his group is “tutorial driven development”. This firmly orients the product around users (who are, for much scientific software, also learners). Matteo (Shearwater) offered expanded advice for starting an open project: focus on documention, tests, and automated project management. This seems especially sensible in the AI era, as it becomes clear that documentation & automation are essential for agents, just as for human collaborators. (And by the way, please can we stop treating them separately?)

Andrea (RWTH Aachen) mentioned that the volume of incomplete or ambiguous issues is a challenge. This presaged Guillermo’s presentation on using agents to manage and maintain open-source projects with a prototype assistant called Steward. Later, Guillermo and Andrea did a quick proof of concept for finding and closing stale issues.

If AI feels more like a threat than an opportunity, Julien’s talk on hardware development was perhaps comforting. It seems that some aspects of software development are easier now, but the impact on (and threat to!) hardware development seems less obvious, to us anyway. Creating, iterating, and modifying hardware solutions is necessarily quite messy, with bits of wire, solder spots, and occasional puffs of blue smoke. Designing PCBs and programming FPGAs is objectively much harder than writing web apps. If you’re worried about your future as a coder, maybe now is the time to try hardware hacking?

The simple lesson

It was striking to hear several people highlight “simplicity by design”. Take Julien: he built a sensitive detector that can be driven across Djibouti, chucked in a boat, dragged up a mountain, and shoved in a volcanic fumarole at 95°. It must be simultaneously robust and hackable with minimal equipment. Every design decision had consequences that, as a field geologist, he understood intuitively. Hence standard industrial tubing, few moving parts, seawater cooling — and running Python on a microcontroller.

Echoing the sentiment, but in software, Collin (USGS) described building composable UNIX-style command-line tools around plain text data files — and users were grateful for the lack of functionality. Nanna (dGB) said early design decisions in OpendTect paid off in allowing the fast addition of tooling for new seismic data formats. Mark (TGS) attributed their success with MDIO to leaning on existing formats and libraries. And in my [Matt’s] own experience, I find people are much more likely to understand and therefore use simple projects. That goes for features too — those that were complex to implement are often the least used.

Myths of open source

Clearly the greatest myth of all is that “open source is free”, or (relatedly) that you cannot build a business around open source. Open source may be free of charge to you, but someone is paying.

Gerard Gorman (Devito Codes) runs the Devito project with an open-core model: the Devito framework is open source, and Devito Codes Ltd runs around this core. Researchers can do fantastic things with the core ‘out of the box’, while the DevitoPRO offering adds software and services. TerraNigma does the same around GemPy, and dGB uses a similar strategy for OpendTect — and has done this for 20 years.

It’s a smart model. The organization does not seek ‘charity’ for the open core, rather it chooses to fund the core itself. This helps separate the concerns, for example when talking to customers and investors, who may be unconvinced about open strategies; it is not their concern.

All this said, one thing about even the small group of maintainers at the workshop was clear: the range of motivations is broad. Openness may be a requirement (as for the USGS), a moral imperative, a commitment to education, expedient for collaboration, or a strategic differentiator. And the range of funding models is correspondingly diverse.

The day was super fun and we think everyone present felt some cameraderie around the drive to collaborate in scientific software. We only hope that when we do it again, we can attract some people and perspectives from outside our cosy clique. One thing we know about doing hard science, nurturing communities, fundraising, and dealing with a lot of uncertainty is that all of it is easier with more humans.

Matteo Ravasi, Shearwater: from pet project to fiscally supported workhorse

PyLOps — Python linear operators — started at Statoil when Matteo worked there. At the time, there was a conscious effort by the company to publish more open-source, and proper documentation, testing, and a trustworthy architecture were hard requirements. Matteo directly pointed to this foundation as a success factor for the project. After leaving Equinor, Matteo guided the project into the NumFOCUS portfolio, and this provided access to Google’s Summer of Code allowing for someone to be paid to accomplish very high impact but difficult to achieve features. Today, PyLOps is used daily for serious work so if you haven’t seen it yet — check out the repo.

Julien Moreau, The NW Edge: open hardware for hydrogen detection

Open hardware does not often attract much attention, certainly not in earth science, but Julien (GitHub) wants to change that. He showed some “data” from a commercial H₂ sensor that must have had a signal:noise ratio close to zero. So, using off-the-shelf components and well-honed hacking skills, he built a water-cooled device that he has deployed in both rural France and in volcanic fumaroles in Djibouti. And the data is beautiful! His lesson for everyone: complexity only adds cost, reduces maintainability, and ultimately excludes ordinary users. The only problem he has now is: how do you actually share a hydrogen detector? 🤔

Collin Cronkite-Ratcliff, USGS: modern public domain software

The USGS was founded in 1879, so maybe it’s natural that some legacy projects only exist as punch cards for Fortran. For example, the gravmagsubs project is partly based on Plouff 1977 (check out the original code listing in the link!). Like the rest of the US federal government, USGS products are mostly public domain — so there’s a strong “open by default” policy and, accordingly, the USGS maintains nearly 1000 public domain software (e.g. under CC-0 licenses), mostly on GitLab. There are packages in hydrology (e.g. MODFLOW) and geophysics, but minerals resources are the hottest current topic (e.g. this R package).

Andrea Morales, GemPy: four pillars in the Temple of Zeus

Andrea is a researcher at RWTH and a contributor to both GemPy and PyGIMLi. GemPy, open-sourced in 2015, is a tool for structural geological modeling under uncertainty, with PyGIMLi a way to connect geology to geophysics. Andrea focused on the challenges of maintaining open-source projects: GemPy is actively developed and it has a lot of users, but they have found it difficult to construct the four pillars of support they need: Terranigma provides consulting and development services on the core tools, RWTH provides research motivation, the community provides ideas and contributions, while funding provides sustainability. These last two are the big challenges — but Andrea is optimistic about the future. GitHub repo.

Nanne Hemstra, dGB: proof that this all works!

dGB Earth Sciences OpendTect is an “open-core” integrated seismic interpretation platform, released as dTect in 2002 and open sourced in 2005. Later, in 2009, it adopted the GPL to help clarify the rights of users and of dGB. Today, it is arguably the most successful open-source project in subsurface, dGB been a persistent innovator in not only software and geoscience but also in business. Nanne highlighted the importance of software architecture and data structures used since the creation of the project, which continue to make it possible to quickly and cleanly add new features and capabilities to the product.

Gerard Gorman, Imperial College London: a case-study of academic-first code

The Devito project is one of the most coherent projects in the computational ggeophysics ecosystem, but it is a relative newcomer having open-sourced in 2016. Echoing Joe Dellinger’s words quoted by me at the start, Gerard’s conviction is that “open source accelerates innovation”. The catch: “sustainability is structurally hard” — and is arguably the unsolved problem in open source. Reflecting on his experience running Devito Codes Ltd, Gerard enumerated some myths that, if unacknowledged, hurt sustainability: “open source is free”, “if it’s popular it must be sustainable”, “we can always fork it”, and “AI can replace maintainers”. Sound familiar?

Matt Hall, Equinor: inside a large open-source publisher

Equinor’s open source efforts, which have been underway since at least 2018, are very much alive. There are around original 500 open source projects on the company’s GitHub account today, covering a wide range of domains, from data assimilation to rock physics to robotics. Although this number of repos is a little anomalous in the world of energy, it’s not unusual at all in the world of technology-centric companies. Most developers in Equinor are well aware that the company has an ‘open first’, and many want to participate in it. This strategy has pay-offs in terms of improved code quality and recruiting — but the main motivation is collaboration, for example with research institutes and business partners.

Mark Roberts, TGS: a modern seismic data format

All geophysicists have opinions about SEG-Y, the venerable seismic exchange format — partly because it lacks a reference implementation, and partly because it was originally designed for tape storage, it has ended up being highly idiosyncratic and poorly suited to modern geophysical computing. Leaning on established technologies, chiefly Zarr and Xarray, Mark described TGS’s cloud-native storage project MDIO, which the company now uses exclusively. For example, all customer deliveries are produced & streamed from MDIO files, using customer entitlement polygons to apply licenses and policies when necessary. GitHub repo.

Mathias Louboutin, Georgia Tech

Referencing David Donoho’s influential 2024 paper on frictionless reproducibility, Mathias pointed out that computational geophysics, while doing well on open code, is perhaps lacking open data. The SLIM research team at Georgia Tech, under the direction of Felix Herrmann, is working on foundation models for seismic — a critical piece of the landscape for the sensible application of generative AI in subsurface. Mathias described a multiphysics model representing latent variables → porosity–permeability → CO₂ → wavespeed → timelapse data where everyone one of those arrows is differentiable. A good way in to this world is probably to start with the InvertibleNetworks.jl library.

John Stevenson, BGS: a QGIS plugin for field geology

John is Research Software Engineering Lead at the British Geological Survey in Edinburgh. (If you have not heard of it, reseach software engineering is the newly recognized role of properly supporting an organization’s scientific and technical computing effort; check out the Society of Research Software Engineering.) Thanks to John and others, the BGS is a big user of the icon open-source GIS QGIS, and John highlighted the usefulness of QGIS as a platform for software delivery with CRAG, for the Collection and Reporting of Associated Geodata. The plugin enables field geologists to connect field observations with photos and sketches to help create georefenced projects. They store everything using the GeoPackage format — more testimony to the superiority of modern open formats over legacy propietary ones. CRAG will be released soon on the BGS GitHub.

Shaowen Wang, KAUST: waveform solutions for any occasion

Shaowen recently finished his PhD as part of the DeepWave consortium at KAUST, working on SWEEP. The marketing headline for SWEEP does sound very cool: “What if every seismic wave equation could live in one memory-efficient, multi-platform package?”. The pipeline uses the Devito symbolic domain-specific language, SeisFlows, the Deepwave toolbox and Shaowen’s own SWEEP tool, the ‘seismic wave equation exploration platform’. The general idea is to support a wide range of use-cases with simple ‘menus’ of options: acoustic or elastic, Torch or Jax, 32-bit or 16, forward model or FWI – then to execute the plan. GitHub repo.

Bob Clapp, Google X: open models & synthetics

A legend of geophysical programming, Bob started off with the timeline of his involvement in SEPLib. I could have happily listed to a couple of hours of these stories, but he compressed 30 years of geophysical programming into about 2 minutes! The story included adventures with SEP model builder and CUDA, and segued into his current project: generating 10’s to 100’s of procedurally generated earth models with corresponding seismic volumes. A sort of “Marmousi generator” perhaps… except I didn’t catch if the model emits a single pseudo-final-stack volume or is even capable of producing gathers. (If it doesn’t today, I’m certain it will soon.) Bob’s team is presenting their work at ICML in Seoul, Korea, at the beginning of July; I expect we will see the first release of these models around that time.

Guillermo Vargas, Shell: agents as maintainers?

Sometimes AI assistance feels anti-collaborative, tending to emphasize personal gains over collaboration. But how we use tools is our choice as developers. In the final act of the day, Guillermo demoed what other patterns might be possible with coding agents. His agent Steward has instructions to help open-source project maintainers with their chores. It checks the ‘open source readiness’ of the repo (e.g. the existence of an open license and a CONTRIBUTING.md file), and makes PRs with fixes. It can also help with issue triage… so what else could it do?

What is the state today?

After a full day of hearing these stories and animated discussion, I felt we had gathered some important data — and maybe anecdata – but not really been able to process much of it. No doubt each will have their own take on what insights these stories and data lead to. But I think a few themes emerged, as well as some ideas what the future might hold.

[UPDATED] I summarized some themes in a follow-up post, check it out.

For the record, the following open source superheroes came to the session: Matteo Ravasi (Shearwater), Julien Moreau (The NW-Edge), Collin Cronkite-Ratcliff (USGS), Andrea Balza Morales (RWTH Aachen), Nanne Hemstra (dGB), Gerard Gorman (Imperial College London), Mark Roberts (TGS), Mathias Louboutin (Georgia Tech), John Stevenson (BGS), Shaowen Wang (KAUST), Matt Hall (Equinor), Guillermo Vargas (Shell), Einar Landre (Equinor), Doug McClymont (Tullow), Tariq Alkhalifah (KAUST).

Lots to choose from

Today there are 13 challenges, some of them more ‘geo’ than others. Here are some of them:

• sequence — Analyse a sequence of rocks to find patterns. Good first problem.
• wireline — Automatically detect bed boundaries in a density log.
• sample-names — Ah, the realities of data-loading!
• prospecting — Combine information from several map layers.
• photomicrograph — An introduction to image processing and analysis.
• regression — A single-variable machine learning regression problem.

One slightly funky aspect of the challenges is that you have to interact with a web API to play, making HTTP GET requests to read your data and submit your answers. The server will tell you if you got the question right or not.

💡 If you want to try one, this Google Colab notebook will help you get started. Or read on for some tips…

Getting started

To get started, you don’t need an account or anything, the server doesn’t know who you are. Just think up a random-ish key and use that to identify yourself (so the server can match your answer to the data you received).

As a quick example, here’s how you can get the data for the first challenge, called sequence:

import requests

uri = 'https://kata.scienxlab.org/challenge/sequence'

params = {
    'key': 'honey badger'  # Choose a unique-ish key.
}

r = requests.get(url, params)
print(r.text)

Now r.text holds a long text string with your input data.

To send an answer, update the params dictionary with the question number and your answer:

params.update({'question': 1, 'answer': 42})
r = requests.get(url, params)
print(r.text)

The server will tell you how you did. In this case: Incorrect. Your answer is too low.

If you get stuck on a question, you can ask the server for a clue by sending a question number but no answer:

params = {
    'key': 'honey badger',
    'question': 1,
}
r = requests.get(url, params)
print(r.text)

If you decide to give it a go, then good luck and have fun! And if you fancy trying to write a challenge — a challenge in itself! — then check out the kata-dev repo.

What does AI look like?

• Blueness (40 out of 72 images, 56%), did IBM start this?
• Brains (14/72, 19%), often split or somehow computerized.
• Humanoids (30/72, 42%), of a very particular type (see below).
• Riffs on The Creation of Adam by Michelangelo (4/72, 5.6%), one of them with the human on the left
• A lot of whatever that holographic display Tom Cruise had in Minority Report

To be fair, one of the images has some information on it. And some contain actual things that are genuinely relevant, probably by accident:

• Humans (14/72, 19%), imagine!
• Circuit boards (8/72, 11%)
• Code (2/72, 2.6%)
• A neural network (1/72, 1.3%)
• Computers (1/72, 1.3%)

It looks pretty bad from afar — look closer, and it gets worse. The creepy robots are nearly always white, with Western European features and completely devoid of personality or facial expressions. Not only do they appear to have genders, but they are overtly sexualized: the “men” are muscular with chiseled jaws, and the women are conventionally beautiful, often with a human face. And I haven’t counted but I’m willing to bet that most of those in assistant roles are “female”.

None of the images contains:

• People being educated, or showing concern, about AI
• A wind farm or solar array being designed with AI
• A doctor or farmer being assisted by AI
• People designing, making or testing AI systems
• People regulating or preventing the use of AI
• Photons with wavelengths longer than 500 nm

It’s not just me

Things have been written about this. AI Myths has a wonderful article about shiny humanoid robots. A report from boffins at the Royal Society, Portrayals & Perceptions of AI and Why They Matter (2017) is predictably erudite, but predates the current hype cycle. Philipp Schmitt (2021) writes beautifully about the subject. He says,

[These images] are harmful to the public imagination, because metaphors matter. They influence how we develop, think of and design policy for emerging technologies, just as they have for nuclear power or stem cell research in the past.

Romele (2022) goes further, albeit in rather technical language, explicitly condemning several of these tropes as unethical:

While the ethics of science communication generally promotes the practice of virtues like modesty, humility, sincerity, transparency, openness, honesty, and generosity, stock images and other popular visual representations of AI are arrogant, pompous, and overconfident. […] The problem with these images is not their lack of reference. Rather, it lies in the way they anesthetize any debate and disagreement about AI.

I agree completely. These images are arrogant. They are becoming part of the popular understanding of AI. They mostly remove actual humans doing actual things with actual AI, and in doing so they collectively tell a huge fib about what AI is, who it is for, and what the issues around it are.

What to do about it

I think we need to stop using these sorts of images immediately. I know they seem ‘right’ — ubiquity is intoxicating. Here, I made you a list of things to look out for, I hope it’s useful:

The good news is, help it at hand! In researching this post, I came across betterimagesofai.org and it’s the best thing ever. Instead of just whining about it like some people, they are “researching, creating, curating and providing Better Images of AI”, and sharing them with the world under open licenses. I love it. And there are some great images in their collection, so check them out.

Alexa Steinbrück / Better Images of AI / Explainable AI / CC-BY 4.0

Unicorn hackers — A digital twin project

Rudy Yuksel (UMaine), Diederik van Binsbergen (NTNU/VUB), Giovanni Aiosa do Amaral (University of São Paulo), Florian Stadtmann (NTNU), Hallgrim Ludvigsen (Enerbits), Olav Kihle (NTNU), Daniel Yang Hansen (NTNU).

With the goal of building a farm-scale digital twin, the team attempted to integrate several datasets and models, and succeeded in building a strong foundation on which to continue iterating. Their system starts with stochastically sampling wind speed and direction from historical data. This feeds a wake model for the wind farm, modeled using FAST.Farm https://github.com/OpenFAST/openfast/tree/main. The result in turn informs an original damage-accumulation fatigue model of the main bearings, in order to predict maintenance needs for each turbine. Finally, a marine safety model searches for an optimal maintenance schedule, using historical wave and weather data. The team also looked at strategies for delaying maintenance by changing operating parameters, for example to reduce the chance of failure during winter.

The team made impressive progress on an ambitious goal, implementing an original method for modeling the bearings fatigue with rotational and actual load.

Energy game project - Design

Fabian Anstock (HAW Hamburg), Ajit Pillai (University of Exeter), Maria Emilia (Mila) Teixeira De Oliveira (Equinor), Olga Usachova (NTNU), Ivana Lapsanka (BAM), Daria Cislo (University of Edinburgh).

This was one of two projects focused on creating “the best game of all time” — an educational energy strategy game that Morten Fredriksen had proposed as an outreach aid. The team quickly adopted the energy trilemma, which captures the tension between the equity, sustainability and security of energy sources, as their framework for the design of this game.

Accordingly, most of the work by this team was analog, collecting open data on average price-per-kWh (reflecting equity), average emissions penalty (sustainability), and the relative reliability and responsiveness (security) of various energy sources relevant to the Norwegian grid: coal, oil, gas, nuclear, hydro, solar, and of course offshore and onshore wind. Scaling the axes of the trilemma to the 0–10 range and multiplying them would provide the final score for a player’s strategy.

Energy game project - MVP

Morten Fredriksen, Lars Petter Hauge, Fredrik Mellemstrand, Robert Dibble (all Equinor), Roman Sukhanov (independent).

GitHub repo is here.

The energy strategy game was more than just clever game design — a second team implemented a prototype game. The player is challenged with meeting the energy demand of Norway for the first week in January 2024. Adding a baseload of nuclear energy is a good start, but nuclear reactors have historically not been very responsive to high-frequency load changes. Wind and solar can provide substantial energy, and at variable but uncontrollable output levels. Gas is cheap and can follow load perfectly, but has a high carbon penalty… and so on.

There was no shortage of feature requests and ideas, the future of this idea is only limited by developer time! And, while there are already sustainable energy games on the web and on Steam (here’s one from ENEL), the down-to-earth realism and real-time data possibilities of this game captured people’s imaginations. Maybe someone should fund this one :)

Thank you

As always, some thanks are due:

• Many thanks to the EERA DeepWind Conference, and especailly John Olav Tande and Daniel Albert at SINTEF for the collaboration.
• The DIGS venue is such a great event space, and Ragnhild Førde looked after us beautifully. Coworking spaces are the best ⭐
• A big thank you to my employer, Equinor, and especially to Lars Petter Hauge, Fredrik Mellemstrand, Mats Andersen and Markus Dregi for extra help figuring out the logistics.

Anonymizers

Lynn Vogel (EBN), Odd Kolbjørnsen (AkerBP), Zana Pepaj (Equinor), Petter Dischington (NPD), Jari Kunnas (Vår Energi).

This dataset, and others like it, contains a lot of names — of people (Knut Hansen), fields (Johan Sverdrup), equipment (Billy Pugh), and report authors (J. Doe et al). Some uses of some names might be considered personally identifiable information; there are also emails, phone numbers, and other data. The team tried applying a combination of NER models (e.g. with Spacy), specialist anonymization pipelines like Microsoft Presidio, and the Azure OpenAI API to the problem, achieving some success. The team crafted some serious ChatGPT prompts to elicit structured NER labeling, and it was very interesting to see how good the model is at this task.

⭐ The jury understood the task right away, and appreciated the difficulty of completing it. They also liked the relatable way in which the story was told.

Embedding enthusiasts

Ryan Cole (Capgemini), Benjamin Kofoed (Equinor), Kristian de Figueiredo Kollsgård (NPD), George Ghon (Capgemini), Bartek Florczyk Vik.

Embeddings of words, sentences, and documents are useful resources in natural language processing. The idea is to cast the corpus into a vector space in which semantically similar entities are close to each other. The team set about creating a clean dataset using various methods from plain regex to transformer-based denoising (TSDAE) in SBERT, then comparing how GPT and SBERT, and a real human geologist, rated the sentences. They were then able to compare sentence similarity using various methods, and score the performance of multiple models, both off-the-shelf and self-trained.

🌟 Special mention Everyone was impressed with this end-to-end data science project, with a pipeline that included data cleaning, and quantitative comparisons between several models.

Zero-shot chatbots

Jörg Peisker (OMV), Doris Winkler (OMV), Dennis Schmidt (OMV), Daan Petri (EBN), Dylan Loss (ConocoPhillips).

The dream many people have when they first meet a smart chatbot is to be able to ask simple questions with ordinary language, and get back exactly what you wanted. Achieving this dream on a custom dataset is, however, challenging; even mundane things like typos (‘wel’, ‘welll’, ‘weel’, ‘wlel’) gave all the teams headaches. This team experimented with various manifestations of the NPD dataset, the LangChain toolkit, cleaning the vector database in various ways, Facebook’s FAISS project for storing vectors, and extensive prompt engineering — which Jörg pointed out, “is definitely a thing”.

⭐ The jury appreciated the sharp focus on real business questions, and the subsequent story of what did not work — and why. The team put a solid, scientific project together.

Knowledge-graphers

Hammad Ali, Catherine Adams, Henrik Busengdal, Anil Dhiman (all Sopra Steria), Thomas Crabie, Adam Hammoumi, Aziz Ben Ammar, Ilyas Tib (all IFPEN), Lars Lukerstuen (Bouvet), Erich Suter (Equinor), and Johannes Åsheim (Fabriq).

The team applied a deep technology stack to exploring the usefulness of graph theory and semantic subject-predicate-object triples in querying large language models. They also explored ways to extract both knowledge graphs and sample questions that would exploit knowledge from multiple documents from datasets like the one we had. With this foundation, the query response pipeline had several elements including:

• Use the Azure OpenAI API to convert a user query to a Cypher query (Cypher is Neo4J’s graph query language).
• Apply the query to the Neo4j graph database, and retrieve the result.
• Simultaneously convert the user query to embeddings and fetch the nearest neighbours from the vector database.
• Use ChatGPT to synthesize a response based on the graph and embedding responses, and providing references for its answers.

Check out the team’s GitHub repo here.

🌟 Special mention The jury was impressed by the project management skills of this large team, all of whom contributed to the end result. Their graph-based approach has clear value and utility in these problems.

Q&A generators

Nolwenn Bernard (UiS), Henri Blondelle (Agile DD), Eirik Morken (Bouvet), Aleksander Jakobsen (Bouvet), Akram Ourir (Sval Energy).

Recognizing that the long-term goal of a domain-specific chatbot will take a lot of work and collaboration, the team set about creating a large database of question-answer pairs. Apart from being a great hackathon project in itself, such an collection would be a valuable asset to the community, for both training and benchmarking models. Drawing inspriation from Stanford’s SQuAD dataset, the team used ChatGPT-4 Turbo with a highly customized prompt to generate 11 200 JSON-formatted candidate pairs from a clean subset of the NPD data. Here’s a simplifed example:

    {
        "Q": "What type of sandstone lies in the 15/12-Beta-
              West reservoir of wellbore 15/12-4?", 
        "A": "Late Jurassic (Oxfordian) sandstone."
    }

Based on a sample of 550 questions, the team estimated that about 80% of the pairs were of sufficient quality to meet their needs. In a nod to accessibility and sustainability, the team estimated the compute cost of generating the collection at USD 36 — plus about 75 person-hours of labour!

Check out the team’s GitHub repo here.

🌟 Special mention The team did a terrific job of explaining and justifying their goal, and the jury were impressed by this impactful contribution to the community.

Metadata extractors

Enrico Riccardi, Wiktor Weibull, Aksel Hiorth (all UiS), Mads Lorentzen (Geo), Sanjay Kamath, Dorra Nouira (both TotalEnergies).

All of the teams faced a noise problem and needed to reduce the training dataset to something models might learn from. This team chose to focus on well entities, focusing on about 10 wells out of more than 4000. The team used the pretrained KeyBERT model to extract keywords/phrases from documents pertaining to a known well, then selected the 18 most relevant and interesting, including things like drilling risk, equipment failure and medical issues. These were combined into custom prompts for ChatGPT-4, which returned structured JSON containing documents from the entire dataset. Once again, the orchestration of an elaborate toolchain being hidden behind deceptively simple outputs — perhaps that sums up all of transformer-based NLP!

⭐ The team presented a nice story, grounded in the perspective of a subsurface professional. This teams work promises to be useful to anyone picking up an NLP project in our domain.

That’s it for this event! Many thanks to all the participants, who worked so hard to learn new things, put them into action, then share what they learned with everyone else. It was, as always, inspiring to see. I’m sure there will be more events like this in the future, so stay tuned and look forward to the next one 🚀

Changelog

• 2023-12-04 — corrected record of exactly what the knowledge graph team did

Projects

Here’s a very quick rundown of the teams that formed at what I believe was the first public LLM-based hackathon in Norway or in the energy sector (AkerBP ran one of their own a few weeks ago):

• Anonymizers — Masking personally identifiable information in public datasets.
• Embedding enthusiasts — Fine-tuning an embeddings model, using cleaner data.
• Zero-shot chatbots — What kind of questions can chatbots answer about the dataset?
• Knowledge-graphers — Extracting a knowledge graph and providing it to chatbots.
• Q&A generators — Generating question-answer pairs for fine-tuning Q&A chatbots.
• Metadata extractors — Automatically pulling metadata from the dataset.

Tomorrow I will put up another post describing the projects in more detail. When it’s up, you can click here to read it!

Credits

It takes a community of organizers to pull off a community event like this. Here’s a probably incomplete list, apologies if I missed anyone (drop me a line!):

• Jesse Lord, (Kadme and Fabriq) for the dataset, which will soon be released under an open license.
• Lukas Mosser, (AkerBP) for the starter notebook and know-how.
• Paul Cleverley, Infoscience for the named entity tags.
• Eirik Haughom and Frode Odinsen, Microsoft for the in-event Azure support.
• The NPD, especially Janke Ro and those involved in FORCE.
• It was my privilege to facilitate the proceedings, a job I am ill-suited for but enjoy anyway 😅 Thank you to Peter for the opportunity!

An invitation

If you’re not quite ready to host one, come and experience one instead. The FORCE Language Modeling Hackathon is happening in Stavanger, Norway, on 30 November and 1 December 2023. I’m stoked to be hosting, with Peter Bormann organizing. The plan is to fine-tune some language models with open subsurface data from the Norwegian shelf. What would you ask a virtual assistant that has read everything about your project?