[Hplusroadmap] Re: Lengthy comments on Intentional Programming

[Bryan Bishop]

On Sunday 18 May 2008, "Chris Gahan" <chris at ill-logic.com> wrote:
> Edge.org interview with Charles Simonyi (interviewer: John Brockman)
> http://www.edge.org/digerati/simonyi/simonyi_p2.html

> SIMONYI: I have been working on what we call "intentional
> programming." It's very exciting. It has to do with professional 
> programming, so it's kind of hard to get into the details. It also 
> relates to the work of evolutionary biologist Richard Dawkins in a 
> fairly direct way. We are trying to create an ecology of abstractions. 
> Abstraction is really the most powerful tool that we have in thinking 
> about problems. An abstraction is a single thing, yet if it is a good 
> one, it can have many applications, even an infinite number of them. 
> So an abstraction may be looked at from one side as a compression of 
> many instances into one generality or from the other side as a special 
> purpose power tool that yields the solution for many problems          

An ecology of abstractions is also what I am creating, in a sense. I 
called it a perception ecology, in that these 'perceptions' are the 
memes, so the connection to Dawkins is preserved. I mention this to 
some extent on recursion.html, but that's more about personal recursive 
self-improvement rather than the perception ecology plans, where we 
have downloadable brains and neocortical minicolumnar structures. 

> The "first law" of intentional programming says: For every abstraction
> one should be able to define an equal and opposite "concretion". So 
> repeated abstraction or parameterization need no longer create "Turing 
> tarpits" where everything eventually grinds to a halt due to the 
> overhead introduced by the layers. In IP, the enzymes associated by 
> the abstractions can optimize out the overhead, based on the enzymes' 
> domain specific knowledge. The overhead associated with abstraction 
> has always been the bane of the very-high-level languages in the past.       

   Pain (overhead) is the cost of the maintenance of boundaries.
So how would the boundaries between these domains be maintained?

> Is programming a form of encryption?, by Charles Simonyi (2005)
> http://blog.intentsoft.com/intentional_software/2005/04/dummy_post_1.
>html

Off the top of my head, yeah. It's social compression, really. You don't 
just suddenly start programming from the Divine Void. You have to have 
documentation, api, you have to know what the words are saying, the 
meaning behind it. And that has to take some serious context. 

> Still, in programming we are in effect editing the encoded message all
> the time. When the intention in the problem changes from A to B, the 
> programmers in fact have to replay the whole thought experiment 
> described above:   
> 1) identify the image of A in the code – effectively all the places
> where A had an effect on the code. This is called code-scattering in 
> aspect oriented programming.  
> 2) encode B in terms of the same implementation assumptions that were
> in force when A and the rest of the code were first coded. 
> 3) edit the code by removing the image of A and inserting the image of
> B. 

Or you can reload the entire attention-space and start from scratch and 
reprogram it. This is impractical for sufficiently large systems, and 
it's the entire refactoring problem. But when there's only a minor 
tweak, it's really useful when the entire system is made up of small, 
individual components, but at the same time also larger components that 
you can work with in chunks. 

> This is true for all the other activities mentioned earlier – for 
> testing, optimizations, or even documentation. Everything that is done 
> to software – other than literal execution on a computer – makes sense 
> only in terms of the original problem. When we look at the code we 
> need to map it back to the problem statement, otherwise any code is 
> devoid of meaning. In other words, a statement:     
>
>    i=1;
>
> could be either correct or it could well be a fatal error. We could 
> decide which only in the context of what the problem was. Programming 
> languages were not designed for the express purpose of encrypting the 
> problem intentions, but they were not designed with the express 
> purpose of retaining the intentions either. In fact, the best means 
> most languages offer for preserving intentions is the 
> trivial “comment” facility with its well known problems.      

The trivial comment indeed ... it preserves very little. The 
documentation problem *sucks*. It's all of that overhead again, from 
the first law of intentional programming. But I don't see how his 
solution to 'intentional programimng' preserves the meaning that we 
struggle to preserve with stupid little text comments (even the 
screaming angry "drunk" comments that we saw in the netscape sources 
just as they were released into the public:)
http://www.jwz.org/doc/censorzilla.html

Tony Smith, Ben Lipkowitz, John Ohno, and I, and a few others have 
talked with each other before about zoomable grammars and other Ted 
Nelson dreams ;-). Zoomable grammars would be where your brain is 
allowed to do some input, and mindbots (brain interfacing agents) 
process all of the excess information that you don't encode into the 
program. As you type, your meanings and intentions could be encoded and 
further elaborated, with your selection of course (i.e., "no, I meant 
this, not that"). Have you ever seen the Japanese word selection 
dialogs on cellphones? Imagine it on steroids, and you might get it.

> The information contents of programs, by Charles Simonyi (2005)
> http://blog.intentsoft.com/intentional_software/2005/03/notations_and
>_p.html

> “You are telling me that instead of a million line program, I could
> have a 20,000 line problem – 10,000 from the specs, and 10,000 from 
> the expression of the software decisions, combined by running the 
> generator. That would be cool because my costs of maintenance, and the 
> costs due to bugs are all proportional to lines. I could even keep up 
> with all the changes. Who is doing this?”     

Yeah, lots of problems can be solved with generators. 

> “The most sophisticated product in the universe uses generative
> techniques.” 
>
> “What???”
> 
> “Life. DNA. It is often called a blueprint but that is a bad metaphor. 
> It is really a generator. Ever wonder how a few measly gigabits can 
> describe a human being, and a few percent of that can describe a human 
> brain? It is because the genome describes how a brain is grown, that 
> is how it is generated instead of detailing what it has in it and how 
> those things are connected like you would have in a blueprint or in 
> source code. It is a proof of your square root rule. A few dozen 
> megabits in the genome can direct the growing of terabits worth of 
> synapses in the brain. Lucky for us because evolution would have never 
> worked with a code that is order n square bigger than what we have! 
> Evolution would have needed a billion squared years to get the project 
> done.”           
> 
> “So your point is that generative programming will take only a billion 
> years. But seriously, what technologies are missing to make this 
> happen?”  
> 
> “We must pay much more attention to the “domains” where the 20,000 
> lines we talked about come from. We need a very flexible and general 
> database for representing the domain knowledge, and we need an editor 
> to capture this knowledge that is acceptable for your subject matter 
> team and also for the programmers. Unfortunately the experience with 
> source code editors is not entirely applicable because the subject 
> matter domain representation is much denser. The domain representation 
> must also be easily processable by the generators.       

I'm not sure what he means here. I think he's approaching this from a 
slightly different perspective than my 'matter/design compiler' and the 
idea of a perception ecology, of capturing all of the excess 
information in order to encode new ways of approaching and compiling 
possibly drastically different contexts to make something that 
functions roughly the same in different scenarios. Whoo, long-winded. 

> Feature X Considered Harmful, by Charles Simonyi (2005)
> http://blog.intentsoft.com/intentional_software/2005/05/feature_x_is_
>co.html

> First, on a practical level, I've also noticed that recently graduated
> programmers are not well educated in the classical theories of Modular 
> programming as described by Parnas among others. Too many programmers 
> seem to think that just by writing classes and methods, and by the 
> appropriate use of private and public attributes, the program will be 
> automatically modular. In fact the opposite seems to be the case. By 
> not having an independent guiding principle for organization, the 
> classes and public attributes become expressions of the programmers’ 
> whims, mere packing crates for the required functionality. The OO 
> programming languages, most notably Java, but unfortunately also C# do 
> not do much to alleviate the situation, by prohibiting the use of 
> procedures outside of classes and not providing information-hiding 
> boundaries other than the class, which hides information but it is 
> also an implementation concept. Here we have again the old programming 
> language conundrum: to get the benefits of a specific abstraction, say 
> information hiding, you have to buy into a specific implementation at 
> the same time. Of course the whole idea of abstraction should have 
> been to leave room for different implementations, that is to factor 
> complexity.                  

I'm not sure that something could be 'ultimately modular' across 
necessarily unknowable future contexts of implementation. Although my 
software on my computer is an excellent example of modularity, in that 
it has applied to my own context without my needing to dig in and 
investigate, that's really only because I'm using an ISA that is known 
to work in a certain way, so the overhead was all in buying into this 
implementation (the P4 under the hood on this mobo) and done once [or 
at least as few times as possible, so I didn't have to write my own 
bootstrapped compiler, etc. etc.]

> So to generalize the “harmfulness” theory, we need to refer to the
> degrees of freedom in the problem statement, in the domain intention. 
> A programming feature is harmful in proportion that it has more
> degrees of freedom than in the domain intention that it is used to 
> solve. By degrees of freedom we mean the potential parametrizations 
> (arguments, properties, attributes, or parameters) for the abstraction 
> and the gamut of their values. This latter is determined by the 
> parameter type: the range of possible parameter values can be small 
> for enumerated types, or very large, for example the “parameter” is a 
> statement list as in a loop. Harmfulness is related to excess degrees 
> of freedoms: more parameters than necessary, or larger parameter types 
> than necessary for the purposes of the problem intention.           

That's interesting; so don't use something like 'goto' because it's 
something that kind of leaves the whole 'domain intention' of the 
original problem hanging wide open, instead of a solid program that 
elegantly solves the problem at hand. 

> What I mean by this is that if we have limited choice of tools, for
> whatever reason, we will rationally choose the most general one so 
> that we can cover a greater set of problems. So the Swiss Army knife 
> is the choice of mountain climbers. It would not be the choice for the 
> master bookbinder, not because the multi-purpose tool would not work 
> in bookbinding, but because its degrees of freedom would mostly just 
> get in the way both by having to open the right blade each time and it 
> could also cause “binding errors” when the general purpose blade 
> slipped while used for burnishing, for example.        
>
> So the better question is how come we programmers have a limited 
> choice of tools at the level of language features? After all we are 
> not limited in creating procedure contents, names, comments, icons, 
> error messages, and so on. Since we are more like master bookbinders 
> than mountain climbers, we should be better equipped.    
> 
> This is further illustrated in our earlier post about the long tail of
> programming languages. All languages at the top were general purpose 
> languages while we found more domain specific languages further down. 
> Today we live in the reality that you have to pick one language and 
> you are stuck with the abstractions of that language. The ability to 
> tune or extend a language to get closer to the domain we solve 
> problems in is yet possible. And of course mixing languages leads to 
> all kinds of complications because our tools do not work well across 
> languages.        
>
> Just as we need a language and a run-time to create the procedures 
> that we want, we could have a language and a run-time to create 
> language features. Unfortunately languages for syntax directed 
> compilation (such as Yacc) did not solve this problem – they described 
> parsers, not language features, they, in effect, had “too many degrees 
> of freedom” which made them very difficult to use. The main 
> difficulty, of course, was the need for the design for the syntax 
> itself – remember, what we set out to create was a language feature, 
> not a syntax. The road toward creating solutions to the meta-problem 
> is truly an arduous one. But until we increase the choice of language 
> features corresponding to the greater variation in applications 
> domains we face, more and more of the features we now have, will at 
> one time or another have to be “considered harmful” as the degrees of 
> freedoms of the features exceed the degrees of freedoms in the 
> domains.              

This is somewhat hard to decode and apply to the programming for 
manufacturing domain that I've been talking about. At the moment the 
design calls for YAML and object serialization so that the objects are 
persistently stored, and then loaded back up and they'd have methods 
which explain how to use {whatever it is that it is - usually a 
manufacturing process, or an object that details how to make it by 
doing abstract steps}; but object-orientation has always seemed like 
something to avoid to me. I wonder how to apply 'language features' 
within the context of a "design compiler" -- assembling a collection of 
functional units to implement some general design, that falls within 
some specification, again the "language feature". Don't know if the 
analogy holds too well.

Since I'm forwarding this to Simonyi, I'll drop some links to this 
manufacturing project: http://heybryan.org/2008-05-09.html (long)
http://heybryan.org/exp.html (dense)
http://heybryan.org/mediawiki/index.php/SKDB
http://oscomak.net/

> Notations and Programming Languages, by Charles Simonyi (2005)
> http://blog.intentsoft.com/intentional_software/2005/05/notations_and
>_p.html

> Instead, we should move further toward the problem. Once we have
> introduced the search primitive that the “while” implemented, we
> should continue asking questions: “What are we doing with this 
> search?” and create a construct for the intention behind the search. 
> To be able to continue creating new constructs, we have to look beyond 
> classes or even aspects.     
>
> We could help this process of creating new constructs move along by 
> making notation and semantics independent. We should be able to simply 
> list the degrees of freedom – the parameters – that are required for a 
> construct. The parameters themselves could be any other constructs 
> with no a priori limitations: expressions, statement lists, 
> declarations, or types. And notations will have to accommodate this 
> flexibility.      

Semantics (parameters (which are constructs), degrees of freedom) and 
notations (constructs). Hrm. That doesn't sound right to me, though I 
agree with the idea of separating what it is that you're building or 
compiling, versus the notation and way that you're writing it out. I am 
sure there is much confusion over which is which. ;-)

> Personal Observations from a Developer, by Mark Edel (2005)
> http://blog.intentsoft.com/intentional_software/2005/03/random_observ
>at.html

> One of the most compelling aspects of Tinkertoy, and one I understood
> only after I had built and used it, was its ability to combine 
> execution of code fragments with viewing and manipulating data. In a 
> desktop window, you could place code and data at will: large or small 
> program structures, individual functions, lists, trees, arrays, etc. 
> Because you could juggle a combination of code fragments and hundreds 
> of results at once, you could use completed and uncompleted code 
> together, performing computations by hand, manually manipulating data 
> structures, feeding results from one fragment to others. You could 
> also build ad-hoc user interfaces by plopping down an appropriate set 
> of functions in a window. It was an environment in which you could 
> explore programming ideas very quickly. Having experienced this 20 
> years ago, I am dumbfounded that we are still programming in 
> text-based languages.             

This sounds like managing an ecology from top-down, and then selecting 
various species to import that have known behaviors into individual 
units to tweak the overall system. It's a more process-centric debugger 
environment, but it also seems to let you build up boxes for 
blackboxing some functionality away and focusing on the specific 
problem. I think the idea here is to build from this process-theoretic 
approach to programming, with the combination of language features, a 
parser generator eventually that would take the requirements and 
boundaries that the programmer installs, and then generate a compatible 
program 'on the other side' (executable code), right?

> What I did write, was code to support a hierarchical
> regular-expression matching framework. The framework carefully 
> constrains what would be a very computationally intensive matching 
> process to what needs to be parsed for displaying exactly what the 
> user is viewing. To the highlight-pattern writer, it feels like free 
> matching, as if they are combining hundreds of expressions, each 
> theoretically examining thousands of lines of code, and applying it 
> effortlessly to their 10,000 line file on each keystroke. The 
> highlight patterns have proved flexible enough to handle all kinds of 
> strange language constructs, thoroughly, and with good error 
> tolerance. More importantly, they are easy enough to use that people 
> are willing to make the effort.           

Automating the use of regular expressions in the programming process is 
an art in itself, I know that much. For years I was manually going in 
and changing different text portions of my code, by hand, just because 
of some small difference I was attempting to implement. Subsequently I 
have this terrible aversion to refactoring. ;-)



________________________________________
http://heybryan.org/