You should use binary lambda calculus for this kind of question, not Python. You also need to specify how memory, storage, network IO are counted against it. Actually, if I was trying to specify a question like this, I'd define an environment in BLC too but only count execution steps for the agent against it.

That way the question would be mathematically well-defined, and there'd be no "spirit of the market".

"Memory used" I would define as either as the maximum length of the program string if using rewrite rules, or I think there's an abstract machine with push/pop operators and then you can define it as the maximum size of the stack.

Then I'd want to know "Do restrictions include all of 1. the initial size of the program 2. the maximum memory used 3. total bits received/sent to the environment 4. Maximum reduction steps, or some subset?"

@Mira I agree this is better in theory, but it has less direct practical relevance. Most people are not going to know how a number of characters in lambda calculus compares to the length of a program in a programming language people actually run programs in. Also I'm not familiar enough with lambda calculus to be confident in my ability to design such a market.

(If I were going to do something like that I'd probably do it via Turing machine instead, specifying a certain number of steps per second, finite tape size, and an input and output channel.)

3. total bits received/sent to the environment

There's no restriction on this. It seems almost certain that any superintelligence interacting with the real world would process huge amounts of data in order to built reliable world-models.

Neural networks can be large, but instantiating a fresh one wouldn't cost any bits to store the data since they are randomly initialized i.e. instantiated at runtime.

Do you allow infinite memory but count the number of touched bytes against the "characters to instantiate"? So "malloc(1000000); memcpy(...)" would technically be a 1 million byte program, even though it takes <100 bytes of source code?

@Mira My intent is to count only the source code, not the amount of memory it uses while running. (With the constraint that it has to fit within a reasonable size of RAM, let's go with 16gb.) Do you see a problem with that approach?

Are you thinking we train an ai and generate say a 100gb file which we then run as an LLM? If so would that file be included? It's not python, it's a huge binary blob.

If otoh we're going to generate it, then currently we'd use lots of C/other libs.

Although if we don't care about speed we could just imagine it were all in python, and would take ~1m times as long to run, which doesn't matter.

This market measures the entirety of the superintelligence, including any external libraries or data files that are needed to run it.

What about libraries like PyTorch, which are commonly used to train ML models but are largely written in C/C++ for efficiency? Python is a glue language, after all.

@TruthGPT Not allowed. And it seems extremely unlikely that the smallest possible superintelligence is a neural network, so I don't expect that to matter.

How much computation does it initially have access to?

@Elspeth I've edited the description, how's that?

Not to be too picky but some external libs are written in other languages. Those seem hard to measure. Also in a lot of cases the actual first AGI won't be perfectly optimized for size and could be cut down a lot.

If this is about the theoretical minimum only, though, those questions don't matter as much. But knowing the theoretical min is a bunch harder than knowing the size of the first one created.

Cool market idea.

@Ernie First AGI is irrelevant, this is about the smallest.

Can you elaborate on how external libraries in other languages work? Python can't run them directly since it's another language, so what exactly is happening there?

@IsaacKing I believe you can compile c or c++ or other languages with special python bindings so they can be loaded by the runtime. I think for example cPickle is such a module https://pymotw.com/2/pickle/ among others like numpy, I think a common json lib, etc. I believe you import some C lib which lets python hook into them at runtime, with subtle & limited interface methods but super fast speed.

There are lots of versions of this but I think even the stdlib has this kind of lib as of my last knowledge.

@Ernie Ok, I think it'd be simplest to just disallow those. The superintelligence and any libraries it uses have to be written entirely in Python.

@IsaacKing hmm okay. That's a valid market, but maybe 90% or more of current Python ai actually uses numpy or pytoch or other external, other language library at the moment.

@Ernie Hmm. Is there an easy way to measure "how many characters those libraries would be if they were written in python"? Or a decent proxy for that?

@IsaacKing unfortunately I can't think of a way. It's libraries all the way down. This applies more to training than to 💨 current LLMs but it might be reasonable to assume that future AGI may sometimes need to retrain, etc, too.

Very silly question.

1. First what is super-intelligence? It's often defined as any agent better at a specific task than 100% of humans. I suppose given unlimited computational power, even a naive depth-first search is rather versatile, agentic, and will be better than 100% of human chess players. Given the condition "can optimize the world for a simple goal" it becomes an even easier question, as even a thermostat minimizing the cost function, delta between the room's temperature and its expected value, is fulfilling this criteria, which can be implemented 1 to 4 LOC, not more complex than an if statement.

2. If we're talking neural networks, is it inference or training? A trained NN, given it fits into the RAM, will certainly run on even a potato, given the generous 100 years, and will likely be just a 100 LOC at most of Numpy invocations, which is common and necessary enough to be considered as essential as the STD. Even for complex architectures, see llama.c, a tiny implementation of Meta's LLaMa model, which even without much cleverness does run quite well if the model is small enough. Again, if you consider an AutoGPT agent enough of a superinteligence (a premise already debatable given how inconsistent its results are), you have luck distilling it into a 100M parameter model and inferencing it using a similar implementation.

3. Do you mean AGI? Well, as far as modern research goes, we don't have any convincing reason it's even remotely on the horizon. LLMs have a very faint grasp of reality and don't really contain internally capabilities like tree search, basic math, or anything else to make them threatening, and overall in their current form seem dead for anything but non-sense generation and demos, at least if you need consistent reproducible and actually useful agentic behavior.