If I learned anything this past year, it’s that everyone thinks these chatbots are going to replace us.

I use these models a fair bit for work as well as for side projects, and so I’m completely of two minds on their usefulness.

Cons

1. I once asked LLaMa-2 30B to summarize a document. It asked me to respond with the summary in json format. I told it that was its job. It sent an angry emoji.
   
2. Last week, we asked both ChatGPT (3.5) and Gemini Pro to name the state with the most counties that are the names of other states. ChatGPT insisted on claiming things like Montgomery are states. Gemini told us answering this would be unethical. (There are 3 states with 4 counties that are state names. Pennsylvania is my favorite.)
   
3. I asked ChatGPT (3.5) how many squares of a rubiks cube do you have to see to know the state of the whole cube, as a lower bound. It told me 54, which is obviously false.
   
4. I asked pretty much every model what ways more, a kilogram of steel, or a billion pounds of feathers. They pretty much universally reply with “They weigh the same!”
   

Pros

1. I asked ChatGPT what song played at college football games goes bah ba ba ba bah buh, ba buh ba ba ba buh. It answered Seven Nation Army.
   
2. I had Gemini provide logo ideas and game name ideas for our game jam. I provided these to our artist as a reference. My wife also did this to name her candle making class. This solves a hard problem in our field - naming things.
   
3. I just think they’re neat.