Design of Experiments (DOE) is a key tool for planning efficient research. Yet, due to limited access to tools and knowledge, the vast majority of experiments are done ‘one factor at a time’ (OFAAT). So, I built and published a free app with Python and streamlit called “DOE Hero” to help.

Introduction

In most companies, experiments are done OFAAT where everything else is held consistent and small changes to complex systems are studied in isolation.

DOE designs have been published in many journals and textbooks; and tools to build your own experiments exist in both enterprise software and open-source R and Python packages. While these resources definitely facilitate good design generation, they are still limited by cost or by specialized setup environments.

My vision was to build a low power, free user interface (UI) that anyone can interact with - fully removing the access limitations to DOE.

Phase One - Building with PyQT

First I tried some open source DOE generators before landing on DOEPY, a libary built on top of pyDOE. Then I built my own class around it to facilitate interaction with a very user focused simple function call schema.

Then I connected that class to a PyQT app I built locally called “DOE Builder.” The app had a nice UI for creating a factor table and picking a design - then getting a csv file of the design and seeing scatter plots and 3d plots of the experimental space.
Here’s a screenshot of that app:

Here’s where I hit the first roadblock - when I tried to package the app and install it on my own pc - the antivirus immediately destroyed the executable. Oh crap. Well that was a good lesson. As it turns out - running anything from an executable requires some kind of signed identity certificate these days because viruses basically ruined that path for everyone. That seriously stymied the development - something I was doing while also employed.

Just look at my commit history from a few years back compared to today - only weekends, and otherwise totally dark.

On top of that - the PyQT version didn’t even approach model fitting - it only built designs.

Phase Two - Building with Streamlit

In my last job I learned about Streamlit - a python focused, server memory supported framework/engine for sharing apps. Plus, if the app is made public, streamlit will even host the computer to run it for you. Plus, with modern AI coding assistant tools, the knowledge gap has shortened so much to where something that would have taken months to learn and then build can be prototyped in a matter of days.

Sounds perfect.

So, using windsurf and having a very clear idea of what I wanted to build - I developed DOE Hero:
https://doe-hero.streamlit.app/

DOE Hero Features

DOE Designer:

  • Supports full and fractional factorial designs (good for catching edge cases, screening)
  • Central composite/response surface and box-behnken designs (optimization)

  • Space filling designs (good for screening / exploration).

  • Provides a saveable JSON for using the tool again after navigating away (saves your factor table)
  • Provides a saveable CSV design table for you to randomize and run
  • If applicable, shows a 3d plot of the design points so you can understand how much factor space is being studied (downloadable as png).

Model Solver:

  • Allows for feature selection and regression equation building.

  • Shows fit statistics and diagnostic plots.

  • Enables interactive prediction with saving both a report and a predictions output.

Next Phases

I’ve started work on a version of DOE hero that includes tableau-like graphing support using PyGWalker and uses PyCaret for machine learning testing. I also recently learned about Optuna, so I’ll be looking into methods of building UI for ML optimization / hyperparameter optimization (HPO).

Third on my mind is incorporating an AI assistant for discussing DOE designs - but that would be something that requires users to supply their own key (risky) or pay for the assistant through some type of transaction I provide. Of course, I could easily also just give folks access through an embedded key that runs out for everyone when it’s used up for the month.