Using Natural Language to Do Science

How Agentic AI Is Changing Physics

James Nightingale

www.jamesnightingale.net

By the end of this workshop you will

  • Consider how natural language is becoming the primary interface for doing data science.
  • Know what agentic AI is.
  • See how AI can act as a scientific research assistant.
  • Discover how agentic AI is transforming scientific software development.
  • Explore vibe coding as a creative endeavour that can enhance scientific insight.
Margaret Hamilton with the Apollo guidance software, 1969.

A health warning

For most of this talk I will drink the AI kool-aid and show you what these tools can do.

At the end, we will come back down to earth and discuss its pitfalls and challenges.

This image may be AI generated.

Remember This?

  • You spot a tiny issue with a matplotlib figure. 🔍

Remember This?

  • You spot a tiny issue with a matplotlib figure. 🔍
  • You change the python code, but it doesn't change the figure. ðŸ˜Ī

Remember This?

  • You spot a tiny issue with a matplotlib figure. 🔍
  • You change the python code, but it doesn't change the figure. ðŸ˜Ī
  • You trial-and-error StackOverflow answers for the next four hours. ðŸ˜Đ

Remember This?

You wish you could just tell matplotlib what you want the figure to look like.

Now You Can!

Using Natural Language

We don't have this problem anymore.

AI lets us use natural language to interface with matplotlib.

Who has used ChatGPT or another AI to help make a science figure?

[Poll the room: hands up.]

Who enjoyed writing matplotlib (or other visualisation library) code and wrestling with its API and syntax?

[Poll the room: hands up.]

What Is Agentic AI?

A chatbot answers

ChatGPT, Gemini and similar tools, in a chat window.

  • Only knows what you paste into the chat window.
  • You run the code, copy back the error, and repeat.
  • Every step goes through you.

An agent acts

Claude Code (Anthropic) and Codex (OpenAI), in a terminal.

  • Runs inside your project folder.
  • Reads every file, runs any command, edits code and checks its own work itself.
  • Asks before doing anything that cannot be undone.

Live Demo

Example 1: One prompt to Claude

Read the source code folders @PyAutoGalaxy and @PyAutoLens and find the PyAutoLens power-law mass model parameterization.

Read the paper on the COOLEST strong lensing standards (https://arxiv.org/abs/2406.08484).

Compare its parameterization to the COOLEST lens model standard by reading the COOLEST source code on GitHub (https://github.com/aymgal/COOLEST).

Go to the PyAutoLens GitHub repository and open an issue comparing the two parameterizations and how they differ.

Flag that this issue is for the Natural Language workshop live demo, give us a winking emoji and close the issue. 😉

Result: github.com/PyAutoLabs/PyAutoLens/issues/820

The part that matters

Example 2: Building a LaTeX Table

Manually Writing a LaTeX Table

  • You copy a LaTeX table template into Overleaf and adapt it to your results. 📋
\begin{table}
\caption{...}
\begin{tabular}{cccccc}
\hline
Run & $L$ (m) & $m$ (kg) & $T$ (s) & $\tau$ (s) & $g$ (m\,s$^{-2}$) \\
\hline
 &  &  & $^{+}_{-}$ & $^{+}_{-}$ & $^{+}_{-}$ \\
\hline
\end{tabular}
\end{table}

Manually Writing a LaTeX Table

  • You open a file with your results. 📂

Manually Writing a LaTeX Table

  • You find the period, damping time and g in the 3σ summary. 🔍

Manually Writing a LaTeX Table

  • You copy and paste them into LaTeX. âŒĻïļ
1 & 0.25 & 0.05
  & $1.0038^{+0.0034}_{-0.0038}$
  & $51.0^{+3.0}_{-3.3}$
  & $9.796^{+0.076}_{-0.070}$ \\

Manually Writing a LaTeX Table

  • You repeat for every column of every run. ðŸ˜Đ
1 & 0.25 & 0.05 & $1.0038^{+0.0034}_{-0.0038}$
  & $51.0^{+3.0}_{-3.3}$ & $9.796^{+0.076}_{-0.070}$ \\
2 & 0.50 & 0.10 & $1.4159^{+0.0042}_{-0.0042}$
  & $106.2^{+13.4}_{-12.1}$ & $9.846^{+0.058}_{-0.064}$ \\
3 & 0.75 & 0.05 & $1.7404^{+0.0071}_{-0.0064}$
  & $54.1^{+4.4}_{-4.8}$ & $9.775^{+0.082}_{-0.072}$ \\
4 & 1.00 & 0.20 & $2.0092^{+0.0081}_{-0.0084}$
  & $132.2^{+8.8}_{-9.9}$ & $9.779^{+0.085}_{-0.083}$ \\
5 & 1.25 & 0.10 & $2.2431^{+0.0055}_{-0.0054}$
  & $113.1^{+10.5}_{-11.9}$ & $9.808^{+0.046}_{-0.052}$ \\

Manually Writing a LaTeX Table

You wish you could just tell LaTeX what you want the table to look like.

One prompt to Codex

The output folder has the fit results for five runs of a pendulum experiment, and paper/main.tex is our paper.

Read the model.results file in each run's folder and build a table in the Results section from them: string length, bob mass, period, damping time and g for every run, with 3 sigma errors.

Write a caption describing the table's contents.

Build the PDF and show me the table.

No More LaTeX!

Now I tell Codex which results go in the table and what it should look like. It pretty much does it.

Using Natural Language to Format a Paper

  • One .tex file per section, the plots, the outputs, and a .agents folder that tells the agent how the paper is written.
  • The plots folder contains the Python scripts that make the plots, and the figures themselves.
  • The agent handles paper formatting, referencing and building the PDF. I read, argue and decide.

Three prompts to your paper

Acronyms "Convolutional neural network" appears all through the paper. Introduce the acronym CNN at its first use, then replace every later occurrence with CNN across all sections, captions and tables.

Restructure Move Methods before Data and merge Sections 4 and 5 into one Results section. Renumber the sections, update every cross-reference, and rewrite each moved section's opening so the paper still reads in order.

Figures Read the journal's figure style guide. Regenerate every plot from its script to meet it: column width, font sizes, line weights and colour-blind-safe colours. Update every figure in the paper and check that each caption still matches its plot.

Who has used agentic AI to help format a LaTeX paper?

[Poll the room: hands up.]

Who enjoyed writing LaTeX code and wrestling with its tables, formatting and syntax?

[Poll the room: hands up.]

Using Natural Language to...

  • Install software: no fighting conda environments, pip conflicts or compiler errors.
  • Manage jobs on a supercomputer: no Slurm syntax, batch scripts or ssh/rsync juggling.
  • Manage code with Git: no memorising commands to branch, merge or untangle a conflict.

Using Natural Language to Do Science

Strong Gravitational Lensing

I study galaxies whose gravity bends the light of a more distant galaxy into arcs and rings, and use that distortion to weigh them, dark matter included.

A strong gravitational lens: light from a distant source galaxy bends around a foreground lens galaxy on its way to JWST.

The COSMOS-Web Ring

Scientists can often look at a dataset and know exactly the analysis they want to perform.

For example, when shown the multi-wavelength James Webb Space Telescope (JWST) observations of the COSMOS-Web Ring, I would think:

The COSMOS-Web Ring

I want to model the F150W and F444W JWST imaging of the COSMOS-Web Ring independently, which are in my local folder dataset/cosmos_web_ring.

Model the lens light with a multi-Gaussian expansion, its mass with a singular isothermal ellipsoid plus external shear, and model the source using a Delaunay mesh.

For speed, run the analysis on the laptop GPU using a JAX optimizer that estimates only the maximum-likelihood solution.

Plot the observed image at each wavelength in the left column, its lensed source model in the second column, its residuals in the third column, and its source reconstruction in the right column.

Doing It By Hand

  • Download the JWST images and store them in a folder on my laptop. ðŸ“Ĩ
  • Find the PyAutoLens example scripts and adapt them to this lens. 📝
  • Upload everything to the HPC and set off the job. ðŸ–Ĩïļ

Even as the developer of PyAutoLens, this would take me at least 30 minutes.

PyAutoLens AI Assistant

Starting an AI Assistant with Knowledge using a Wiki

Everything the agent learns, from papers, results and conversations, is written into a wiki in natural language.

Each new task builds on that knowledge, so the agent knows more about your science every time you use it.

The PyAutoLens Assistant Wikis

The assistant starts with three wikis of knowledge:

  • Literature: my 400 favourite strong lensing papers.
  • Documentation: the PyAutoLens ReadTheDocs and API.
  • Tutorials: end-to-end science examples.

[Live Demo]

PyAutoLens AI Assistant

The COSMOS-Web Ring, imaged by JWST.

Your starting promptCopy
I want to model the F150W and F444W JWST imaging of the COSMOS-Web Ring independently, which are in my local folder dataset/cosmos_web_ring.

Model the lens light with a multi-Gaussian expansion, its mass with a singular isothermal ellipsoid plus external shear, and model the source using a Delaunay mesh.

For speed, run the analysis on the laptop GPU using a JAX optimizer that estimates only the maximum-likelihood solution.

Plot the observed image at each wavelength in the left column, its lensed source model in the second column, its residuals in the third column, and its source reconstruction in the right column.

The Result

The assistant fits both wavelengths and plots the data, lensed source model, residuals and source reconstruction: F150W on top, F444W below.

Published PyAutoLens lens models of the COSMOS-Web Ring in F150W and F444W: data, model image, residuals and source reconstruction.

There are over 100 JWST Strong Lens Images

A mosaic of strong lenses from the COSMOS-Web Lens Survey (COWLS), imaged by JWST.

Your starting promptCopy
There are over 100 JWST strong lenses in my local folder dataset/cowls, one subfolder per lens.

Fit every lens with the same model I used for the COSMOS-Web Ring: F150W and F444W modelled independently, the lens light with a multi-Gaussian expansion, its mass with a singular isothermal ellipsoid plus external shear, and the source using a Delaunay mesh.

Run them on the HPC, one lens per job, and tell me when they have all finished.

Making Science Accessible (Outreach & EDI)

The COSMOS-Web Ring, imaged by JWST.

Your starting promptCopy
I want to use the PyAutoLens Assistant: https://github.com/PyAutoLabs/autolens_assistant First clone that repository, cd into it and follow its AGENTS.md.

I'd like to understand how gravitational lensing works using the JWST image of the COSMOS-Web Ring that ships with the assistant. Show me the picture, explain what we are looking at, and walk me through fitting a lens model so we can measure the mass inside the ring and see how well the model reproduces the observations. Pitch it at my level: ask me what my background is first. Explain what we are doing as we go, and let me ask questions or change the analysis along the way.

Try it on my website: www.jamesnightingale.net

Loop Engineering & Code Optimization

How I used to make PyAutoLens fast

  • ⏱ïļ Time every block of code in the part I wanted to speed up.
  • ðŸĒ Find the slowest part.
  • ðŸĪ” Think whether I could make it faster.
  • ✏ïļ Change the code, rerun it, and time it again.
  • 🔁 Repeat, one idea at a time, for days or weeks. ðŸ˜Đ

But honestly? I really enjoyed this process. 😄

Code as sculpture ðŸŽĻ

Like a sculptor, I worked the code by hand, one chip at a time, until the fast version emerged from the block. ðŸ”Ļ

This image may be AI generated.

Code as sculpture ðŸŽĻ

Like a sculptor, I worked the code by hand, one chip at a time, until the fast version emerged from the block. ðŸ”Ļ

After a few months of using agentic AI, I realised this process would soon be obsolete: the sculptor would have to hang up their tools. ðŸŠĶ

This image may be AI generated.

Three key aspects of this task

  1. ✅ Trust: the code runs end-to-end, so you can instantly confirm the answer is correct.
  2. 🧠 Expertise: optimizing code often draws on domain expertise beyond our own knowledge base. For me, this was linear algebra, spatial geometry and CPU/GPU programming.
  3. âģ Time: the process is time intensive and trial and error, and sooner or later you need to get back to doing science.

This makes it the perfect problem for agentic AI and loop engineering. ðŸĪ–

What is loop engineering?

Instead of turning the loop yourself, you design the loop and let an agent turn it. 🔁

I can have an agent run this loop: it makes the code faster and I'm happy. 😊 But when it finishes, the next agent starts again from scratch. 😕

The loop is not complete. What do I need to do to complete it? 🙋

Closing the loop with a wiki 📚

Remember how we gave the PyAutoLens assistant its knowledge? Wikis.

Now the agent writes up its work in a profiling wiki:

  • ⏱ïļ its timings
  • 📊 its results
  • ðŸ’Ą its interpretation of the findings

When the next agent starts the loop again, it draws on the knowledge of every previous agent. 🔁

[Live Demo]

Two realisations

  1. 🔁 Run the loop 24/7. PyAutoLens just keeps getting faster: over 10× so far! 🚀
  2. 💎 Stay in the loop. Ask the repository, in natural language, what research it found and how the code sped up.

From sculptor to gardener

The sculptor ðŸ”Ļ carves every detail by hand.

The gardener ðŸŒļ doesn't sculpt the flower. They create the environment and conditions for it to blossom.

Loop engineering is gardening. Setting up the loop is preparing the soil and the environment. ðŸŠī

This image may be AI generated.

Vibe Coding

Have you noticed visualisations getting better, or people using websites and new interfaces to interact with their results?

[Poll the room: hands up.]

[Live Demo]

Using Natural Language to Do Science

Natural language is becoming the primary interface for doing science.

  • 💎 Natural language provides the interface.
  • ðŸĪ– AI provides the capability.
  • 🧑‍🔎 The scientist provides the understanding, direction and judgement.

Today: figures and papers 📊, an AI research assistant 🔭, loop engineering ðŸŒą and new ways to share results 🌐. The syntax was never the science.

AI Pitfalls

⚠ïļ Progress without understanding

With AI it is very easy to make rapid progress without understanding what you are doing. 🏎ïļ

And with the pressure to publish and perform, we may take the fast option. 📈

This impacts early-career researchers (ECRs) the most. 🎓 I had 10+ years of matplotlib, LaTeX and HPC, and wrote PyAutoLens, before AI: senior scientists could stand to benefit the most. 🧠

  • 🚀 Senior scientists can finally accelerate all the projects they could never wrap up.
  • 😟 ECRs are left questioning their purpose, when any science analysis can be done in a week.

Weightlifting or forklifting? 🏋ïļðŸ—ïļ

  • 🏋ïļ Weightlifting: you go to the gym to work out. You could use a forklift to lift the weights, but what's the point?
  • 🏗ïļ Forklifting: if you need to move 10 slabs of concrete to a building site, use a forklift.

With AI, decide beforehand whether you are weightlifting or forklifting.

As supervisors, we need to actively encourage our ECRs to weightlift. 💊

Analogy after science-fiction writer Ted Chiang, writing in The New Yorker: "Why A.I. Isn't Going to Make Art" (2024).

Will our jobs be automated? ðŸĪ–🧑‍🔎

The thought has crossed my mind: "Do I want a PhD student, or do I want a bazillion Claude tokens?" ðŸĪ”

But the more I use AI, the more I want more domain expertise, not less. 🧠

To study disk galaxies with strong lenses (something I know little about), I don't want more Claude tokens. I want an expert on disk galaxies to work on the project with me, with natural language as the interface. ðŸĪ

As of November 2026, I don't think AI agents are replacing the role of the human. They are changing it. 🔄 (Though I'd like the models to stop getting better quite so fast! 😅)

Unresolved challenges

  • 🔒 Data privacy. Your data, code and ideas go to a giant tech company. We must have an open-source AI ecosystem emerge.
  • 🌍 Environmental impact. GPUs use a lot of energy. But AI made my code faster, reducing its energy use and emissions. Hopefully AI can lead to efficiencies and technological breakthroughs. ⚡

Claude Team for Scientists

We have 25 free Claude subscriptions for the year.

I am keen to lead the rollout, but we should agree on how we approach it first.

The syntax was never the science

Turning an idea into an analysis has always meant translating it through matplotlib, LaTeX, Slurm and Python. That syntax was only ever the interface between your thinking and the computer.

Now the interface is natural language. You say what you want in the words you already think in, the computer does it, and you keep the understanding, direction and judgement.

Margaret Hamilton with the Apollo guidance software, 1969 (public domain, Wikimedia Commons, restoration by Adam Cuerden). Interfacing with a computer used to mean this much code.

A chatbot answers. An agent acts: it has tools, and it loops. The loop is the whole difference. It sees the error you would have pasted, and the file you would have opened, without you in the middle. You talk to it the way you would brief a capable student. Gemini CLI, Cursor and Copilot are the same idea with different wrapping.

Five instructions. Two code bases, a paper, an external standard and GitHub. Nothing below this was typed by a human.

Follows Karpathy's "LLM wiki" idea: a curated wiki the model reads and extends, rather than retrieval over raw PDFs.

Stand-in: this is the PyAutoLens fit published in Mercier et al. 2024 (A&A 687, A61), shown as what the assistant produces for this prompt. The CC BY-SA credit line on the image must stay.

Analogy after Ted Chiang, "Why A.I. Isn't Going to Make Art", The New Yorker, 2024.