Example: "How many three-digit integers contain at least one 2?"

Elegant permutation solution:

9*10*10 - 8*9*9 = 252

It's also fun to write a program:

three_digit=[]
for i in range(100,1000):
if "2" in str(i):
three_digit.append(i)
print(len(three_digit))

It's a less trivial problem if you make it: "How many three-digit EVEN integers contain at lest one 2?"

(but it's still trivial in code. Just add "and i%2==0")

Example: "How many three-digit integers contain at least one 2?"

Elegant permutation solution:

9*10*10 - 8*9*9 = 252

It's also fun to write a program:

three_digit=[]
for i in range(100,1000):
if "2" in str(i):
three_digit.append(i)
print(len(three_digit))

It's a less trivial problem if you make it: "How many three-digit EVEN integers contain at lest one 2?"

(but it's still trivial in code. Just add "and i%2==0")

To start a popular and annoying tangent to this conversation... this is EXACTLY the sort of problem LLMs are terrible at solving. (unless they have been fed multiple correct versions of the solution)

The LLM will tell you a numerical answer with all the confidence of a god and it will be totally wrong. This understandable if you think about the way that these responses are generated, but to me this alone shows the limits of the system.

To start a popular and annoying tangent to this conversation... this is EXACTLY the sort of problem LLMs are terrible at solving. (unless they have been fed multiple correct versions of the solution)

The LLM will tell you a numerical answer with all the confidence of a god and it will be totally wrong. This understandable if you think about the way that these responses are generated, but to me this alone shows the limits of the system.

@futurebird And really, a limit of human psychology that unless you're really really deep in your soul understanding how it works, the confidence will absolutely convince you,

Today I completed my planning for my summer course for math teachers "Computer Science for Math Teachers"

But I'd love to know of any additional topics I ought to include. Main goal is to get the rest of the math faculty comfortable enough with code that they use it in a natural way as one of many problem solving techniques and teach this to our students.

When my lesson plans have a CS component other math teachers often skip it just because they don't feel comfortable.

@futurebird I tried it on a local LLM and it followed the reasoning: determine how many 3 digit numbers there are, use complementary counting for numbers without a 2, and subtract one from the other. I suppose it might appear on its training set, or it might have generalise from similar problems.

@bri_seven

I think because this is the kind of problem that a computer can be used to solve with great confidence and accuracy people just assume that LLMs are doing what I've done in writing a program to literally count all of the numbers that meet the condition.

It's not an absurd error to make... but the LLM is doing no such thing!

@futurebird i have noticed that in newest versions of chat-GPT it can indeed write a python program, run it, and give you the results. They charge extra for that capability. also still no guarantees the program it writes is correct

@bri_seven

So, they've added python to it to some degree do you think? That's interesting.

@futurebird What are your thoughts on teaching them eg. Haskell? To me, Lambda Calculus feels like a natural gateway between math and programming

@futurebird it refers to it as “analysis”, it has special specific UI in which you can examine the python program and its output, and optionally download it to run on your own machine- a little like how it presents ui for other external services like image search results, image generation, image description- the llm itself doesn’t do those things but makes it seem like it is

Example: "How many three-digit integers contain at least one 2?"

Elegant permutation solution:

9*10*10 - 8*9*9 = 252

It's also fun to write a program:

three_digit=[]
for i in range(100,1000):
if "2" in str(i):
three_digit.append(i)
print(len(three_digit))

It's a less trivial problem if you make it: "How many three-digit EVEN integers contain at lest one 2?"

(but it's still trivial in code. Just add "and i%2==0")

@futurebird Is 8*9*4 just a typo, or are you checking to see how alert we are?

9 × 10 × 10 – 8 × 9 × 9 = 252 (correct)

9 × 10 × 10 – 8 × 9 × 4 = 612

@futurebird it refers to it as “analysis”, it has special specific UI in which you can examine the python program and its output, and optionally download it to run on your own machine- a little like how it presents ui for other external services like image search results, image generation, image description- the llm itself doesn’t do those things but makes it seem like it is

I think one ought to be able to bounce in and out of both ways of seeing the problem seamlessly.

Use brute force to verify your theory. Use theory to make better brute force.

@futurebird one thing I've always appreciated in the intersection between coding and computer-science/mathematics is, when you can use a naïve approach initially, to get answers fast; but then have to switch to a more sophisticated algorithm to have any hope of scaling. I'll see if I can come up with some specific examples…

@bri_seven @futurebird
Presumably constraining the LLM output to valid python programs at the sampling stage, potentially giving a misleading impression about its capability

@eigen @futurebird

Back in the day when computers were rare, if I had to show people who were completely new to coding one simple thing you could program super-easily if you weren’t too worried about efficiency, and then refine to get better scaling, I always took them through a few versions of listing the prime numbers up to N.

Example: "How many three-digit integers contain at least one 2?"

Elegant permutation solution:

9*10*10 - 8*9*9 = 252

It's also fun to write a program:

three_digit=[]
for i in range(100,1000):
if "2" in str(i):
three_digit.append(i)
print(len(three_digit))

It's a less trivial problem if you make it: "How many three-digit EVEN integers contain at lest one 2?"

(but it's still trivial in code. Just add "and i%2==0")