[Hplusroadmap] Re: metarepository / SKDB & OpenVirgle: Fwd: Mostly for Bryan: Subversion, git, darcs; JSON, Yaml; Pointrel; distributed architectures
Bryan Bishop
kanzure at gmail.com
Sat May 10 21:46:02 CDT 2008
---------- Forwarded Message ----------
Subject: Mostly for Bryan: Subversion, git, darcs; JSON, Yaml; Pointrel;
distributed architectures
Date: Saturday 10 May 2008
From: "Paul D. Fernhout" <pdfernhout at kurtz-fernhout.com>
To: OpenVirgle <openvirgle at googlegroups.com>
Just for reference on what Bryan talks about with git (which I think is
a better choice for SKDB than darcs btw based on popularity):
"The Future of Subversion"
http://tech.slashdot.org/article.pl?sid=08/05/09/1528250
"As the open source version control system Subversion nears its 1.5
release, one of its developers asks, what is the project's future? On
the one hand, the number of public Subversion DAV servers is still
growing quadratically. On the other hand, open source developers are
increasingly switching to distributed version control systems like Git
and Mercurial. Is there still a need for centralized version control in
some environments, or is Linus Torvalds right that all who use it
are 'ugly and stupid'?"
For the record, the more I think about Bryan's idea of using git (or
something similar) as a platform to do distributed work like with SKDB,
the more I like it. And I think as a technology approach, it can let me
easily make distributed Pointrel-based systems, as the first version of
Pointrel checked in to the code project merges easily as all changes
are made at the end. :-)
By the way, on yaml,
http://yaml.org/
I've liked the idea of JSON in the past, but not used it:
http://www.json.org/
"JSON (JavaScript Object Notation) is a lightweight data-interchange
format. It is easy for humans to read and write. It is easy for
machines to parse and generate. It is based on a subset of the
JavaScript Programming Language, Standard ECMA-262 3rd Edition -
December 1999. JSON is a text format that is completely language
independent but uses conventions that are familiar to programmers of
the C-family of languages, including C, C++, C#, Java, JavaScript,
Perl, Python, and many others. These properties make JSON an ideal
data-interchange language."
A comparison somewhat in YAML's favor of the two:
http://en.wikipedia.org/wiki/JSON#YAML
"Both functionally and syntactically, JSON is effectively a subset of
YAML."
By the way, YAML is a little like this proposal of mine (easier in some
ways, harder than others): RFC: Lisp/Scheme with less parentheses
through Python-like significant indentation?
http://groups.google.com/group/comp.lang.lisp/msg/6d55f641ea635982
(Lisp code is essentially just a tree of symbols, numbers, other trees,
etc.).
Still, anything that is encoded that way could be encoded in the
Pointrel system or RDF. :-) But not really by hand with the Pointrel
system, since it is more a library for managing triples then a way to
specify them easily as text. The intent was always to do that via a
full programming language. I'm kind of now thinking more and more of
the value of simpler (safer) languages like Yaml/JSON for
specifications (the Pointrel web version used Python which is unsafe,
but it did different stuff with that). Still, there is something to be
said for a potentially sandboxed but full featured language like
Jython. Different strengths and weaknesses.
So, for a distributed OSCOMAK (or SKDB :-) this is a thought:
Content including stuff written in one or more computer languages
Pointrel
git & content interpreters
Various OSs
Each layer rests on the one below, and some of the content defines the
GUI and simulation tools, same as was done with the Pointrel web
version Mike played with, which pulled programs from the repository and
evaluated them to produce web pages. :-)
Of course, the useful core of git in this application perhaps could
probably be replaced with a few dozen pages of code of Python using
Pointrel (not all of git, just a simple merging process).
And for the content to include cross-platform programs easily, the JVM
is nice.
So then, for easy portability of GUIs and simulations: :-)
Content (including Jython code and
JSON/YAML/indented-Lisp/Semantic/etc.)
Pointrel
Bootstrap Jython program
Jython
JVM
In practice, this brings me back to the first stuff I checked into the
code project and Mike downloaded :-) But now thinking it could
eventually benefit from adding distributed versioning code which I am
implicitly using Subversion for now. But in my model, that is just
more content. And also that it should support "safer" content like
JSON, YAML. and/or Semantic MediaWiki-like tags.
Bryan is apparently implicitly looking at GUIs via ikiwiki generating
cross-platform HTML for a web browser. So he goes in a different
direction; ikiwiki requires Perl. Perhaps he is also really targeting
GNU/Linux implicitely, not Mac/Win/Linux as I was with the JVM? I'm not
sure.
So he has:
Content, some in YAML
ikiwiki
Configuration files and dedicated SKDB application code?
Perl & git
GNU/Linux (but maybe others)
Maybe this is not correct as to what Bryan intends, but it is a first
cut. And so everyone can see the side-by-side comparison of software
stacks.
But that *still* leaves the issue of what to encode. :-)
Which is the hard stuff at this point IMHO. :-)
So, right now I am still thinking mostly about content and metadata to
put in the centralized Semantic MediaWiki on the server, not
infrastructure (this note excepted :-).
This is all by way of saying I'm still paying attention to what Bryan is
saying (and not saying :-).
--Paul Fernhout
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google
Groups "OpenVirgle" group.
To post to this group, send email to openvirgle at googlegroups.com
To unsubscribe from this group, send email to
openvirgle-unsubscribe at googlegroups.com
For more options, visit this group at
http://groups.google.com/group/openvirgle?hl=en
-~----------~----~----~----~------~----~------~--~---
-------------------------------------------------------
________________________________________
http://heybryan.org/
---------- Forwarded Message ----------
Subject: Re: Mostly for Bryan: Subversion, git, darcs; JSON, Yaml;
Pointrel; distributed architectures
Date: Saturday 10 May 2008
From: Bryan Bishop <kanzure at gmail.com>
To: openvirgle at googlegroups.com
On Saturday 10 May 2008, Paul D. Fernhout wrote:
> Just for reference on what Bryan talks about with git (which I think
> is a better choice for SKDB than darcs btw based on popularity):
> "The Future of Subversion"
> http://tech.slashdot.org/article.pl?sid=08/05/09/1528250
> "As the open source version control system Subversion nears its 1.5
> release, one of its developers asks, what is the project's future? On
> the one hand, the number of public Subversion DAV servers is still
> growing quadratically. On the other hand, open source developers are
> increasingly switching to distributed version control systems like
> Git and Mercurial. Is there still a need for centralized version
> control in some environments, or is Linus Torvalds right that all who
> use it are 'ugly and stupid'?"
>
> For the record, the more I think about Bryan's idea of using git (or
> something similar) as a platform to do distributed work like with
> SKDB, the more I like it. And I think as a technology approach, it
> can let me easily make distributed Pointrel-based systems, as the
> first version of Pointrel checked in to the code project merges
> easily as all changes are made at the end. :-)
I came back after finishing this email to comment here. You're still
taking the approach of asking when we'll get our hands down and
dirty .. but it's really quite simple. Just start a git repository,
dump content in it via wget, or via your own ~/cache/ (like mine (it's
approaching its third gigabyte of papers, plaintext, etc.)). That's
all. :-)
> Still, anything that is encoded that way could be encoded in the
> Pointrel system or RDF. :-) But not really by hand with the Pointrel
> system, since it is more a library for managing triples then a way to
> specify them easily as text. The intent was always to do that via a
> full programming language. I'm kind of now thinking more and more of
> the value of simpler (safer) languages like Yaml/JSON for
> specifications (the Pointrel web version used Python which is unsafe,
> but it did different stuff with that). Still, there is something to
> be said for a potentially sandboxed but full featured language like
> Jython. Different strengths and weaknesses.
Before I get contaminated by the rest of the email, let me jot out the
overall structure that I have had in the back of my head for a while
now:
== Physical layer ==
* hardware/peripherals
** microprocessors/controllers
** instruments (arms, rollers, "robots")
[[This is all standardized, see below].
* internet
== Digital layer, offsite wrt Avg Joe ==
* various 'distributions'
** debian, gentoo, redhat meets debfab, fabgen, fabhat ;-)
* main project websites, communities
** skdb, metarepo, oscomak
(BTW: metarepo sounds immediately obvious to me, FWIW).
Super offsite:
* amorphous network of projects on private servers
* crawlers that aggregate latest git repos into skdb
== Digital layer, onsite wrt Avg Joe ==
* Standardized distribution software
** debian, gentoo, etc. I like debian, so I'll be using it.
* agx-get -- download from the metarepo
** metadata & semantic data are via YAML
* autospec - put packages together, re: manufacturing
** GNU units
* material inventory management
* matter compiler
** this one is *tough* -- but I have some really *tough* guys helping me
out on this one. We can do it. A good challenge to have, IMHO.
* cupsd but for more than inkjet printer jobs
** job generator
*** to some extent already done by prewritten scripts from SKDB, python.
**** this is because of Py-YAML. Object serialization is good.
** job scheduler
** machinery orchestration
> So, for a distributed OSCOMAK (or SKDB :-) this is a thought:
>
> Content including stuff written in one or more computer languages
(1) Formalized content: plug and play ;-)
(2) Natural language content: have to read it and suggest some new
programs to make it do something interesting instead of just sitting
there and being dry.
> Pointrel
Package relationship management? Maybe, but this could also be done in
hardcode too, since the package maintainers would have to modify this
anyway.
> git & content interpreters
> Various OSs
OSes for the microprocessor hardware; and now 'fabhat' for the OS of the
entire manufacturing operating system [obviously still an OS; just an
extension of the drivers and so on, as unix was originally meant to be
used [like with mainframes]].
> Each layer rests on the one below, and some of the content defines
> the GUI and simulation tools, same as was done with the Pointrel web
> version Mike played with, which pulled programs from the repository
> and evaluated them to produce web pages. :-)
>
> Of course, the useful core of git in this application perhaps could
> probably be replaced with a few dozen pages of code of Python using
> Pointrel (not all of git, just a simple merging process).
Hm. That might be possible, but I'm skeptical, since merging in terms of
git pulling is not the same thing as merging different project branches
entirely, since functional specifications and requirements can change
to an extent that can't be automatically detected. Although I wouldn't
be ideologically opposed to programmers writing tools to help automate
mergers like that. I just think the reusability might be slim. Might be
wrong. Not a big deal, it seems like something that wouldn't take too
long to code up?
> And for the content to include cross-platform programs easily, the
> JVM is nice.
Yep, that's what I've been considering for the instrumentation problem.
Since you could have a mechanical arm that looks a certain way, and I
could have a mechanical arm that is something totally different, how
the hell do we make this all work as intended without forcing everybody
to ever use a non-exact instrument to rewrite all of the code that they
download? :-(
> So then, for easy portability of GUIs and simulations: :-)
> Content (including Jython code and
> JSON/YAML/indented-Lisp/Semantic/etc.) Pointrel
> Bootstrap Jython program
> Jython
> JVM
I guess. My experiences with java have never been good. Always kinda
clunky. Not the rapid experimentation that a scripting platform would
provide. I think I heard once that Sun came out with a java bytecode
platform, a physical microprocessor that ran the bytecode directly, and
even *then* there was performance issues. ;-)
Am I wrong?
> Bryan is apparently implicitly looking at GUIs via ikiwiki generating
> cross-platform HTML for a web browser. So he goes in a different
> direction; ikiwiki requires Perl. Perhaps he is also really targeting
> GNU/Linux implicitely, not Mac/Win/Linux as I was with the JVM? I'm
> not sure.
Hm. Well. First, the reason ikiwiki is interesting is because it uses
git under the hood, and the fact that it's web accessible means you get
to let all of the little fishies out there bite and see it from the
web. It's just a way to get them interested. When you download a git
repo, you are downloading what the wiki is, basically. So the
presentation hooks (ikiwiki is one of them) can easily be modified. I
think that an underlying YAML syntax is close enough to wiki syntax to
play around with, or even using ikiwiki plugins to show YAML as YAML,
and let separate wiki-formatted files be there for web presentation
purposes. Dunno. Frankly, I'm hoping graphviz will have some
visualization ideas [maybe to be integrated with pointrel and dia].
> So he has:
> Content, some in YAML
> ikiwiki
> Configuration files and dedicated SKDB application code?
Possibly - backup stuff, spidering/aggregation, that sort of thing.
> Perl & git
Not necessarily perl. Rewriting ikiwiki should be easy.
> GNU/Linux (but maybe others)
Re: my focus on GNU. I mentioned above the issue of OSes. Basically,
you're going to need an operating system for the manufacturing
installation. A distribution, like 'fabhat'. The apt-get model is there
for a reason. So Windows is largely out of the question (no way in hell
we want viruses here, but this isn't my main reason). I hear Mac OS X
has a linux kernel under the hood now. Does it have an apt-get equiv?
> But that *still* leaves the issue of what to encode. :-)
>
> Which is the hard stuff at this point IMHO. :-)
Infohoard into zip files and git repositories all of the information and
do rapid prototyping. Mike was nearly doing this, but 'solar energy'
isn't actually a package, it's more like a unit.And then you two come
along with your licensing and rights stuff ;-).
> So, right now I am still thinking mostly about content and metadata
> to put in the centralized Semantic MediaWiki on the server, not
> infrastructure (this note excepted :-).
Just put as much content as you can; for example, let's say that we were
going to make a microprocessor. Actually, this is an excellent example
thanks to OpenCores. So you'd go fetch all of their files, their
manuels, documentation, FAQs, news articles, etc. Then we'd package
them up with the metadata stuff. Put them into git repositories. They
are probably already in a versioning system. So maybe we can run some
cvs-to-git programs on them (if they aren't already using git).
Then there's the matter of programs related to the OpenCores project
themselves. This would include stuff like gEDA, and eventually the
manufacturing code that would tell the JVM (or its equivalent?) what to
do to make it [in general [subject to how discussions on the matter
compiler turn out]]. In python.
- Bryan
________________________________________
http://heybryan.org/
-------------------------------------------------------
________________________________________
http://heybryan.org/
More information about the Hplusroadmap
mailing list