Senior Software Developer
|
And here's another picture. This time the FeWork was created to read "Howdy doody", and then revised so it reads "Good bye." You can see the bert canopy developing at the upper right and the sensor canopy on the lower right. I'm not sure what to call the thing that's growing in the upper left, but it's part of the canopy as well.
|
Senior Software Developer
|
Two more pictures. The first again shows a simple stored secret, but the canopy has expanded dramatically because another work has been stored which refers to the first.
The second shows the same structure from the point of view of the other work. I think the "boo" off in the left-hand corner is the name of the lock which has been placed on the work. (Just say "boo" to get in.)
|
Senior Software Developer
|
Here is a picture of a structure where one work transcludes another. Transclusion is essentially quoting by reference -- the transcluding work actually contains a chunk of the work it is transcluding. The DspLoafs indicate a displacement in the range of text being included, and you can see that the same text has been included by two different OVirtualLoafs.
I should note that the colors in these pictures aren't precisely right. Sensors are gold, the bert canopy is brown, and that thing in the upper-left corner is green, which I believe are close enough to the traditional colors. However, red is supposed to be for content and blue for history, but I'm not using them consistently here.
|
Senior Software Developer
|
One last picture. The BeGrandMap. I like how the Ent looks like it's on the road to a deep, dark forest.
(Upon further review, it looks like the BeGrandMap will start using its Ent if you ask it to store more than 16,384 characters. So I think I'll have to try that to see what that looks like.)
|
I'm really not sure what to say to all this. What little I actually know about coding lets me know that what you have is crazy. Impressive, too. And if it actually will store all the data in the world, well, sign me up for one now!
Other than that, this has been incredibly entertaining to read so far. Keep at it, PostMonster!
|
Senior Software Developer
|
As part of my continuing efforts to understand the thing, and possibly harness its powers for my own purposes, I have been exploring what it takes to build a "front end" for it. A front end is the device you use to tell the machine your secrets and retrieve them later. The mechanisms for making a front end are not clear, and the literature is scant on this important topic. (For example, there are references to "fm," or "frontend magic," but the only part of the machine identified as such is the Thunk.)
This research has led me to a mildly disturbing realization about the contraption: You can put secrets into the machine, but you can never get them back in their original state. Or rather, you can, but the designers of the machine really would prefer you didn't.
For example, let's say you tell it the secret "Mary had a little lamb." When you give the secret to the machine, it is in the form of a string of characters. The machine, as noted previously, translates this string into a fiendishly complex data structure.
Now, let's say you want to retrieve that string of characters, perhaps because you would like to send them to someone else. The machine will not let you do this, at least not easily. What you get back is an Edition of a Work, which contains RangeElements. You most certainly do not get back a string of characters.
So how do I get my string of characters? They're definitely in there somewhere. You need a tool which will manipulate the data structure for you at arms length. In other words, a Waldo. Previously I had thought the Waldoes were extinct, but I believe they may have morphed over time into a family of things called Wrappers.
The wrapper we want is called FeText, which the literature unhelpfully describes as a device which "handles a integer-indexed, contiguous, zero-based Edition of RangeElements." Whatever. It's got controls that let you insert, move and replace stuff. Not strings of characters, of course. That would be too obvious. No, you make an FeText from an array of data and then insert that FeText into an FeText, which gives you a new FeText with the newly inserted data.
That's all well and good, but how do I get my string back out of this FeText thingy? Well, you ask it for its contents. Simple! Right? Noooo. As mentioned above, you get back an Edition of a Work which contains RangeElements. Grr.
What you do is this: Ask the Edition to "retrieve" its contents. This gets you a Stepper. As long as you haven't done anything fancy to your string of characters (like moved stuff around, or inserted new text), this stepper will, when asked, give you "theOne." This gets you something called a FeArrayBundle. And that is the thing which actually contains your string of characters.
Seriously, what the hell?
In all the complexity of this contraption, there is no one single simple device which you can use to just get the string of characters. This is a huge, obvious, glaring, gaping hole. The absence of this tool can only be deliberate.
This thing is supposed to store all knowledge. And by that I mean, permanently store all knowledge. You are always and forever expected to never want that knowledge in any other context except that provided by the machine.
Here's why. Let's say you told the machine "Mary had a little lamb," and then told it "whose fleece was white as snow." That's two separate secrets. Then you make a new secret which combines the two. "Mary had a little lamb whose fleece was white as snow" is a new secret, but the machine knows that it is composed of two separate secrets. If you were to ask the machine for that new secret and it just told you "Mary had a little lamb whose fleece was white as snow" and it didn't also tell you about the two original secrets and how to find them and how to find all other secrets which refer to or are composed of parts of those two secrets, and also what other secrets refer to or are composed of parts of the new secret, well then the machine is simply not doing its job.
So when you get that Edition from the FeText, and you get the Stepper from that, you have to step over the RangeElements to display the new secret. You are most emphatically not supposed to simply decompose that structure into a simple string of characters because you'll lose all that valuable context.
From the point of view of the contraption, once you tell it secrets, those secrets should never leave. There is no "Save As..." button. There is no "Export." Want to share your secret with a friend? The madmen who built this thing desired that your friend ask the contraption for the secret directly herself. The contraption knows all, what it doesn't know is not worth knowing, and there are no other ways of knowing than through the contraption.
When I say the designers intended it to hold all the knowledge in the world, they meant that in a startlingly bold way that doesn't quite sink in at first.
|
Senior Software Developer
|
A new picture demonstrating range transclusion, without the canopy. Wherever you see a branch (SplitLoaf, OVirtualLoaf, OPartialLoaf), there is a distinction between regions in the coordinate space. The "myIn" branch is everything within the distinction, and the "myOut" is everything outside the distinction. I believe an OrglRoot represents a particular edition of a work.
|
Senior Software Developer
|
Hmmm. Now that I understand what I'm looking at a bit better, a few of Those Which Need Removal may need to be resurrected. Based on their names, FeCompletionDetector, FeFillInDetector and TransclusionRecorder might actually be useful. If they work. I haven't yet investigated whether or not they do.
I may also need to investigate SplayEntLoaf, simply because it's a loaf. And I'm interested in loafs.
(I also need to take back what I said earlier about Fluids and Emulsions. They're too deeply embedded to simply rip out. No matter how much I'd like to have them gone....)
Loafs? I saw that title and was expecting an _entirely_ different kind of thread...
Rooby-Roo?
|
Senior Software Developer
|
Based on their names, FeCompletionDetector, FeFillInDetector and TransclusionRecorder might actually be useful. If they work. I haven't yet investigated whether or not they do.
I may also need to investigate SplayEntLoaf, simply because it's a loaf. And I'm interested in loafs.
Nope. Turns out those are all useless vestiges.
|
Senior Software Developer
|
Names.
Names are important. How you refer to a thing helps determine what you think you can do with that thing. And the names of things within the contraption have changed many times over the years of its construction. What follows is speculation.
Those which have been referred to as "Stamps" are now "Editions" (and actually BeEditions).
"Bert" is not an ernie, rather, it is a "Work."
Waldoes may have turned into Wrappers. I am not at all confident about this.
Tonight's theory about the relationship referred to as "hcut": This stands for "history crum upper trace." This is almost certainly wrong, but I like it.
|
Senior Software Developer
|
Trawling through the archives over the holidays, I discovered this gem:
"Each inner loaf just holds a distinction region. The left child is all the data inside the distinction, the right child is all the data outside the distinction."
An amazing insight. No wonder the ent is so spare.
And it turns out that what I meant by that is, "no wonder the code for the ent is so spare." The contraption itself is amazingly, horribly, fiendishly complex once it gets going.
Is it true that the basic concepts of this theory were laid down by a sociologist in the late fifties mid sixties ???
One of my friend is also a software developper and he told me that what you are describing was first used for a completely different purpose. I just wanted to know if my friend told the truth.
|
Senior Software Developer
|
The original visionary coined the words "hypertext," "hypermedia," and "micropayment". I don't know that I've ever heard him described as a sociologist, though he has referred to himself as "poet, philosopher and rogue." I've been a fan of his work for almost 30 years; he's been at it for nearly 50.
As to its original purpose, well, the contraption isn't really being used for any purpose right now except when I make it pose for snapshots. Perhaps someday it will actually be used for its original purpose, or something like it. (I know there's at least one project here at Paizo for which this contraption would be absolutely perfect.)
I've ever heard him described as a sociologist, though he has referred to himself as "poet, philosopher and rogue." I've been a fan of his work for almost 30 years; he's been at it for nearly 50.
Ted Nelson as defined by Wikipedia.
Well, wiki might be wrong (again...)
|
Senior Software Developer
|
In the case of Ted Nelson, I think it's probably better to find out about the man from his own words, rather than what's been written about him and his works, which is why I linked to what I think is his personal site.
Beyond that I won't say more because we've never met and anything I could say would be sheer speculation. (Such as my telling of the tale of the madmen and their visionary earlier in this thread -- all of that is from secondary sources and is probably wrong. But, you know, artistic license, telling a good story and all that. Mea culpa. :-)
|
Senior Software Developer
|
Those which have been referred to as "Stamps" are now "Editions" (and actually BeEditions).
"Bert" is not an ernie, rather, it is a "Work."
Waldoes may have turned into Wrappers. I am not at all confident about this.
Tonight's theory about the relationship referred to as "hcut": This stands for "history crum upper trace." This is almost certainly wrong, but I like it.
I have been informed (thanks, Deanster!) that these speculations are mostly wrong.
Except one. Bert did become Work.
An "hcut" actually probably indicates a distinction. (Or, and this here is speculation, based on absolutely no exploration through the workings of the contraption, a "history cut".)
Nobody remembers precisely what became of the Waldoes. But apparently they were too complex and weren't really necessary, so they probably did go extinct. I now think Wrappers are an entirely different family.
PS: When Deanster said it took 9 colors to draw the ent, I thought he was kidding. I thought you could probably draw it with four (blue, red, brown and green). But he proved it to me yesterday, in crayon. The whole experience broke my brain for several hours.
PPS: He did say he didn't quite remember what the ninth color was for, so maybe he really was pulling my leg a teensy bit. But I don't think so.
|
Senior Software Developer
|
Those which have been referred to as "Stamps" are now "Editions" (and actually BeEditions).
What I now know about stamps:
Stamps are a slightly lower-level tool that happens to also be used for Editions. I only recall vaguely (though I might be able to figure out from the code), but I think that Stamps are the identity abstractions that is used to navigate the canopies. Basically, a version compare would stamp one edition, and then navigate down the other edition looking for stamped loaves (loaves that refer upward into a canopy node that has the desired stamp). Long-lived stamps were also useful to application code (e.g., stamp the "canonical version" of a document), so the facility may be exposed out in the API.
Or, rather, what I've been told about stamps. I'm pretty sure I don't really know it just yet.
|
Senior Software Developer
|
Tumbling through the names
As I've mentioned before, the creators of the contraption were extravagant with the names they bestowed on the various parts of the beast. Snarf. Ent. Turtle. Flock. Bert. Orgl. Crum. Waldo. Enfilade. And, of course, Loaf, among many, many others.
These names got coined and evolved over four decades of development. When they built a new thing, they gave it a new name, partly to help solidify its literal uniqueness in the world. So, Fossils and Recorders and OTrees and HTrees and the Granfilade and the Poomfilade and the Bert Canopy and the Sensor Canopy and the DagWood.
By the time they began work on the version of the beast I'm currently exploring, I think they started wondering how well these names would resonate with those who they hoped would use their creation to build even more wonderful things. So they changed some of the names once again. Orgl became Edition, Bert became Work.
It's a little sad to see those names go by the wayside since they'd been used for so long, but I think saying you have an Edition of a Work is much clearer than saying you've got an Orgl of a Bert. And anyway, the names live on via the internal structures of the thing, so that's nice. (For example, we still have OrglRoots and the Bert Canopy.)
However, in at least one case, I'm thinking about turning back the clock.
This project has for decades had the concept of an infinitely expandable number space. Like 1.25.0.0.0.33333.92.42.2021.0.5. And the math to go with it -- you can take two of these things and add them, subtract them, do all sorts of nifty things with them. They've long been used to carve out a way of addressing the smallest chunks of the secrets given to the contraption, even across multiple machines.
When I was much, much younger than I am now, I read about these mystical things and loved them. Part of the reason was what they were called: "tumblers." The name evokes many things, from acrobatics to the clicking of the inner workings of a lock (or a bank vault!) as the key is rotated into place. The idea that something so serious and powerful could have such a playful name appealed to me.
So what are they called today? "Sequence." Um. Yeah, sequence, whatever. Sure, the literature defines the term as "an infinite sequence of integers," so I suppose the name makes some sense.
But to me it's still a tumbler.
|
Senior Software Developer
|
Let's revise a work by moving a chunk of text around. We've already got the work via the process described earlier. This time the text is "Mary didn't have a little lamb." We're going to move the word "didn't" so the revised work reads "didn't Mary have a little lamb."
First off, we need to enable recording of history for this work. So we set FeWork's history club to the public club, which means anyone can read the history. The FeWork is really just a front for the BeEdition, so the BeEdition actually sets the club on itself.
Next, we're going to get a FeText, which is a Wrapper on the FeWork's edition. As before, getting a FeEdition "on" the BeEdition involves getting a label. This time, however, the FeLabel is "on" a real BeLabel, so the BeLabel introduces the FeLabel to its myFeRangeElements (which is a set). The FeEdition remembers myBeEdition and myLabel, and the BeEdition introduces the FeEdition to its myFeRangeElements. The FeText sets the edition as myEdition, and now we've got an FeText.
We need to tell the FeText about the region of text we'll be moving. In this case the region is characters 5 up to but not including 12, so we make a new IntegerRegion with those boundaries. The contraption has a rather sophisticated understanding of regions -- they can have holes in them, they can extend to infinity, etc., and can be transformed in various ways. So the IntegerRegion knows whether it starts inside (it doesn't in this case) and how many transitions it has (two). The notation for this region is: "[5, 12)".
The FeText first needs to validate that we're moving text into a valid position. To see what positions are valid, the FeText needs to get its count, so it asks its FeEdition for its count, which asks the BeEdition, which fobs the job off onto its myOrglRoot. This ActualOrglRoot is also a slacker, so it asks its myO for its count. The myO here is an OExpandingLoaf (yay for loafs!) and it turns out that it too is going to pass the buck to its myRegion. And guess what -- it's an IntegerRegion. Aren't you glad we just learned about IntegerRegions?
So the IntegerRegion returns its count and we can get back to the important business of moving text around, right? Wrong. First the IntegerRegion needs to double-check that it's not infinite, because then it'd be silly to ask it for a count. It's considered finite if its bounded both below and above, and this one is. In fact, the notation for this region is "[0, 30)", so the job of counting is easy enough that I'm going to skip it.
Now we can move some text. Here's where we get really, really glad the FeText is going to handle moving this text for us. Internally the contraption is all about regions and intersecting them and unioning them and mapping them onto one another and transforming them in various ways. Here's a 30,000-foot overview of what FeText is about to do for us:
Get the region of the edition's domain which has moved, by intersecting the entire domain with the region we're moving. Get the region representing what's left behind by subtracting the moved region from the edition's domain. Now, take a deep breath, as the FeText is going to construct a new edition for us based on its current edition, transformed by a mapping. The mapping consists of a) the region of what's left behind that intersects the region before the position where we're inserting the moved text, combined with b) the region we're moving transformed by a mapping based on the position where we're inserting the moved text, combined with c) the region of what's left behind intersected with the region after the insertion position, transformed by a mapping of the moved region unioned with the region starting from zero up to the insertion position.
Here is the region of what's left behind after extracting the text we're moving: "{[0, 5), [12, 30)}". You can see it's more complex, and consists of two separate regions.
The mapping looks like this: "CompositeMapping(IntegerMapping(-5) on [5, 12), IntegerMapping(0) on [12, 30), IntegerMapping(7) on [0, 5))". If you break that into its pieces, you can see that it's a CompositeMapping of three separate IntegerMappings. The first indicates a -5 displacement on the [5, 12) region, which means we're moving the text 5 spaces to the left. The next is a 0 displacement for [12, 30), which means that text isn't moving. The last is a +7 displacement for [0, 5), which means that text is moving seven characters to the right.
(If the word "displacement" reminds you of DspLoaf, give yourself a cookie. You deserve it! I think I may need to apologize to DspLoaf.)
The beauty of this contraption is that its machinations involve all those regions. Remember, regions can be disjoint and very complex. You could make a region which consists of just the vowels in this messageboard post, and getting its complement would get you all the consonants. The transformations that the FeText is performing for us can just as easily be applied to "didn't Mary have a little lamb" as to the 92nd draft of War and Peace, the one where Tolstoy for some reason inserted seventeen (non-consecutive) pages of "All work and no play makes Homer something something." The FeText cares not how complex the underlying structure is, all it has to do is a handful of mappings and transformations and its job is done.
FeText is kind of a special case in that all it knows how to handle are IntegerRegions in an IntegerSpace. But the contraption knows about several other kinds of spaces, like RealSpace and CrossSpace. And of course, SequenceSpace (which I may yet rename back to TumblerSpace).
Next, we'll step through the construction of an edition based on the original "Mary didn't have a little lamb" edition, transformed by the mapping the contraption just made for us.
|
It's a little sad to see those names go by the wayside since they'd been used for so long, but I think saying you have an Edition of a Work is much clearer than saying you've got an Orgl of a Bert. And anyway, the names live on via the internal structures of the thing, so that's nice. (For example, we still have OrglRoots and the Bert Canopy.)
Obfuscation is a beautiful and necasasry thing!
When dealing with abstractions I think it best to name things abstractly.
"I have an Edition of a Work" excites all kinds of biases and preconceived notions whereas "I have an Orgl of a Bert" forces a higher degree of conciseness that at once indicates a specific actuality while maintaining a high degree of other-worldliness, thus applicability.
Truly miraculous software should not be digestible by the casual observer.
|
Senior Software Developer
|
As I recall, the admittedly tepidly-named "Sequence" space was a space whose Positions were an ordered sequence of positions in some other space. Thus, TumblerSpace is properly Sequence<Integer>. I was pretty sure that Tumblers were still around, since everybody loved the name too :) (But they were certainly not as central as they used to be.)
It looks like the project may have been heading in that direction, but in the version I have, a Sequence is a tumbler (ie, has hard-coded support for tumblers and only tumblers).
There was another parameterized space, maybe called Box?, which would describe a multidimensional space. For example, an Edition in the space Box<IEEE64,IEEE64,IEEE64> could describe a CAD drawing. Add an integer dimension for part numbers and you can describe parts in a Cad drawing. Or link objects in the space to associated documents.
I think this became CrossSpace ("Represents the cross of several coordinate spaces"). BoxAccumulator, BoxStepper and BoxProjectionStepper survive, but I think they probably need to be generalized. [edit] Actually upon quick review it looks like they are already generalized.
|
Director of Sales
|
| 1 person marked this as a favorite. |
But to me it's still a tumbler.
Tumblers can be very useful!
For example, when reviewing this thread last night, I realized the need to procure a FeBot<tl> of OGin and parse it into discreet OParts. I found a Tumbler with XIce to be perfect for this use.
Now my head feels like an OExpandingLoaf.
|
Senior Software Developer
|
On to the loafy goodness involved in revising a work!
First, we'll get a new FeEdition created by transforming the current edition by the mapping. The FeEdition is created "on" the myBeEdition transformed by the mapping. To transform the BeEdition, first it checks to see if the mapping is a Dsp -- if it were a Dsp, then the BeEdition could hand the work off to its myOrglRoot. In this case, however, it's a CompositeMapping, which is not a Dsp.
And now we run smack dab into the middle of Fluids and Emulsions. The contraption uses these to create what are essentially global values that persist during the workings of a particular part of the machine. The contraption says, the CurrentTrace must be thus-and-so while I am performing this operation, and uses "fluid binding" to accomplish this. I have given fluids and emulsions the stinkeye before, but have come to the conclusion that the machine is far too dependent on them to easily tear them out.
The CurrentTrace is going to be set to a successor to myHCrum's hCut. The myHCrum is a HistoryCrum, and its hCut is a TracePosition. The TracePosition asks its myBranch to "create after" the TracePosition. The myBranch is a BranchDescription, so it first checks to see if its last position equals the trace's position, which it does. So the BranchDescription will return the next position. To do this, it increments its last position, and returns a new TracePosition based on itself and the last position. We get back a BoundedTrace which remembers the BranchDescription as its myBranch, and takes the position as its myPosition. And now we have the CurrentTrace.
We also need a CurrentBertCrum for the duration of this edition transformation process, but we simply make one, unrelated to any other BertCrums that might already be out there. Now we can begin transforming.
The BeEdition starts by getting a domain (which is a region) by asking its myOrglRoot for its "simple domain." The ActualOrglRoot simply returns its myRegion. Next, we need an OrglRoot which will eventually contain the transformed secrets, so we make one based on the mapping's range space. The CompositeMapping's range space is IntegerSpace, and we end up with an EmptyOrglRoot, which makes a new HBottomCrum for its myHCrum. Since the OrglRoot is also a type of OPart, it also makes a new SensorCrum for its mySensorCrum, and asks that crum to add a pointer to the OrglRoot. Interestingly, the SensorCrum completely ignores the OrglRoot's request to be remembered individually, and instead just increments its myRefCount.
Now that we have the domain and an OrglRoot, we'll start stepping over the mapping's simple mappings. For each, the contraption will extract the common region between the domain and the simple mapping's domain (where they intersect). If the common region isn't empty, we'll get the simple region's Dsp and use that to combine the OrglRoot with a copy of the BeEdition's myOrglRoot corresponding to the common region, transformed by the simple region's Dsp.
First, we get the common region where the domain and the mapping domain intersect. The IntegerRegion first checks to see if either it or the other region has no transitions. Here they both have transitions, so we get edge steppers from each. I think I'll skip ahead past the whole intersecting regions thing for now, and take it for granted that we do in fact end up with a new IntegerRegion. (The process involves getting an IntegerEdgeAccumulator and checking transition counts, and edges and stuff like that.)
The common region is: "[5, 12)", which isn't empty, so we can ask the mapping for its Dsp, which is an IntegerMapping(-5). Next, the BeEdition's myOrglRoot is asked to create a new OrglRoot corresponding to the common region. This region is "simple", so we'll just step over its distinctions and copy each one.
And just as we're about to run into our first splay, I'm going to pack it in for now and continue spelunking later.
|
Senior Software Developer
|
It's splay time! And the original reason for the Loaf Question!
The ActualOrglRoot is going to copy its first distinction, which is a region. There is a note in the literature that the region must be a valid thing to store as a split, but there's no validation. To copy a distinction, an OrglRoot "splays" a region into its own subtree as close as possible to the root. A splay operation makes each child completely contained or completely outside the region, and returns the number of children completely in the region. If the result of this splay is 0, we'll get an EmptyOrglRoot. If there are two children, we get the ActualOrglRoot we started with. Otherwise, we make another ActualOrglRoot consisting of the OrglRoot's myO's inPart and the intersection of the OrglRoot's region and the distinction region.
The ActualOrglRoot asks its myO to perform the splay. Its myO is an OVirtualLoaf, a type of InnerLoaf, and also a type of OExpandingLoaf. I'll be referring to it here based on which family the behavior comes from.
First the OExpandingLoaf checks to see if its myRegion is a subset of the distinction region. If so, then the splay will return 2. (Determining whether or not an IntegerRegion is a subset of another again involves edge steppers and lots of "is entering" checks which I'll skip for now.) The OExpandingLoaf's myRegion is "[0, 30)" and the region we're splaying (against? on? for? about?) is "[5, +inf)", so it's clearly not a subset. It does, however, intersect, so the OExpandingLoaf will do an "actualSplay" instead of all this pretend splaying.
The actualSplay operation first gets a Pair of crums by asking the OVirtualLoaf's sensor crum to "expand." If the CanopyCrum has two children, it returns them. In this case, it doesn't, so it makes two children and sets itself as their parent, and updates the CanopyCache. In this case, the CanopyCache didn't already know about the children, so it ignores the request. The Pair of crums will be an HUpperCrum and a SensorCrum.
Next, the OVirtualLoaf creates two new OVirtualLoafs cleverly named "tmp1" and "tmp2". (Better names would probably be "in" and "out".) The first is constructed around the region where the OVirtualLoaf's domain intersects the region, and tmp2 is constructed around the region where the OVirtualLoaf's domain intersects the region's complement. Each new OVirtualLoaf gets a new HUpperCrum based on the Pair of crums just created, and uses the SensorCrum as it is. The HUpperCrum is constructed by taking the paired HUpperCrum "(1389.3)" and getting its hCut (a TracePosition, "1389.3"), and getting (1389.3)'s bertCrum.
Again, I'll skip the actual intersecting of the regions, but the end result for tmp1 is a new IntegerRegion: "[5, 30)". The OVirtualLoaf is created with the SharedData from the loaf that's doing the splaying (the SharedData is the thing that actually contains the string of bytes "Mary didn't have a little lamb"), the region, the HUpperCrum and the SensorCrum. The region for tmp2 is "[0, 5)".
We've reached the step that caused the Loaf Question in the first place! The OVirtualLoaf that's doing the splaying has to now "become" a completely different kind of loaf. It needs to become a SplitLoaf, because it now designates a distinction. So my cheesy hack comes into play, finally. Instead of actually "becoming" a SplitLoaf, it creates a new SplitLoaf based on the region, tmp1, tmp2 and its hcrum, and then sets that as its private loaf. From here on out, whenever asked to do anything, the OVirtualLoaf will hand the task off to its private loaf.
The SplitLoaf uses the region to indicate its split (or distinction). Tmp1 is its myIn, tmp2 its myOut. (Everything in the myIn is inside the distinction, everything outside the distinction is in the myOut.) It uses the hcrum it was provided, but computes a join between the inloaf's sensor crum and the outloaf's sensor crum to get its own sensor crum. (I'll skip computing the join for now, but the literature says that the idea is to "find a CanopyCrum that is an ancestor to both crums. The second crum is added to the canopy in a pseudo-balanced fashion.") The final step of creating the SplitLoaf is to ask its myIn and myOut to add itself via addOParent. The two loafs hand the job off to their hcrums, and another join is computed, this time in the Bert canopy.
I think this is a good a place as any to pause. At this rate, it may take a week. But it'll be an interesting week! (To me, at any rate.)
|
Senior Software Developer
|
The second distinction, and splaying a SplitLoaf
The next distinction the ActualOrglRoot is going to copy is (-inf, 12), and again we're going to splay that region, and again the responsibility is passed onto the OrglRoot's myO, OVirtualLoaf(, 1381.3). The OrglRoot's region is [0, 5), and the OVirtualLoaf's region is [5, 30).
The OVirtualLoaf is also an OExpandingLoaf, and again it first checks to see if its region is a subset of the distinction region. It's not, so it checks to see if its region intersects the distinction. It does intersect, so it's time to do an actualSplay. Once again, we get a pair of crums by asking the OVirtualLoaf's sensorCrum to expand.
As before, two new OVirtualLoafs are created. The in region is [5, 12), and the out region is [12, 30). The OVirtualLoaf now becomes a SplitLoaf.
The result of this splay is 1, i.e., there is one child completely within the region, so to copy the distinction we make a new ActualOrglRoot from the SplitLoaf's in part (OVirtualLoaf(, 1381.3)) and the intersection of the ActualOrglRoot's region [5, 30) and the distinction (-inf, 12). The intersection is [5, 12). We're done copying the distinctions for this simple region.
Now we're back to the BeEdition, which as you may recall, is in the process of creating a new edition consisting of the common region where the edition's domain and the mapping domain intersect. We've now got that edition. The next step is to transform that edition by a Dsp, the IntegerMapping(-5).
A transformed ActualOrglRoot will be made from the ActualOrglRoot we just made, with its myO transformed by the Dsp and the Dsp's "ofAll" of the ActualOrglRoot's region [5, 12). To transform a Loaf, you make a new InnerLoaf based on the Loaf and the Dsp. And that loaf is a DspLoaf. The DspLoaf adds the transformed loaf as its myO, and does the usual CanopyCrum stuff which I'll explore in depth later. The Dsp's ofAll of the ActualOrglRoot's region is [0, 7).
So now we have the transformed oroot: ActualOrglRoot([0, 7), (IntegerMapping(-5))). The BeEdition now steps forward to the next simple mapping, which is an IntegerMapping(0) on [12, 30). The common region here is [12, 30) and the Dsp is (0) since nothing in this region is moving. Again, the current ActualOrglRoot will be combined with a copy of the BeEdition's myOrglRoot region in common with the common region, transformed by the Dsp.
Once again, the region is simple, so the ActualOrglRoot steps over its distinctions and copies them, and to do that it performs a splay. The first distinction is [12, +inf), and the myO, formerly an OExpandingLoaf, now a SplitLoaf, performs the splay for its limit region of [0, 30). The limit region intersects the distinction, so an actualSplay is performed.
This is the first time we've seen a SplitLoaf splay, and it turns out that splaying a SplitLoaf is non-trivial. When a Loaf splays (on? in? around?) a limit region, it has to make its children completely inside or outside the region. If the SplitLoaf's region partially intersects the limit region, then you have to swap, rotate, interleave and/or split the child loafs. There are nine separate transformation cases that need to be taken into consideration.
I'm not sure I quite follow the notation in the literature, but here's an example of the fourth type of transformation: in=1, out=0, we'll return 1 (number of children in the region), and the operation is a rotate right, or ((A* B) C) -> (A* (B C)). The simplest types of transformations are 1, 7 and 9, which all do nothing. The most complex looks to be the fifth type, which is called an interleave, and the notation for that is ((A* B) (C* D)) -> ((A* C*) (B D)).
We'll see what kind of transformation we're dealing with shortly. First, the SplitLoaf asks its myIn to splay (on? in? around?) the region where its mySplit [5, +inf) intersects the limit region [0, 30) for the distinction region [12, +inf). Then it asks the myOut to do the same for the complement of its mySplit.
myIn is a former OExpandingLoaf, now SplitLoaf, and the limit region intersects the distinction region, so it too will do an actualSplay. Oy. Fortunately its myIn is our friend OVirtualLoaf(, 1389.3) so hopefully all this splaying won't go on forever. Anyway, it will be splaying a distinction of [12, +inf) and a limit region of [5, 12). The regions don't intersect, so it returns 0, indicating there are no children inside the region. The myOut, an OExpandingLoaf in turn does its splay around the region [12, +inf) and the limit region [12, 30). Its region [12, 30) is a subset of the limit region, so it returns 2, indicating both its children are within the region.
Back to the myIn at the beginning of the previous paragraph, the second SplitLoaf we've encountered so far. It's now asked its in and out to splay, and we've got an in count of 0, and an out count of 2. According to the literature, this is a type 3 splay, or swap #7 : (A B*) -> (B* A).
The out is greater than the in, so the first thing it will do is swap the children. The literature says, "Swapping the children reduces the number of cases. This way, if this crum is partially in a region being splayed, the part contained in the region resides in the left slot." Swapping the children involves switching the myIn and myOut, and changing mySplit from (-inf, 12) to its complement, [12, +inf). This completes the splay operation, and the myIn returns 1.
Next, we splay the myO of the first SplitLoaf we encountered (it's still in the middle of its splay). It's splaying the region [12, +inf) with a limit region of [0, 5). Its myRegion is [0, 5), which doesn't intersect [12, +inf), so it returns 0.
The first SplitLoaf now has an in count of 1, and an out count of 0. That's a type 4 splay, described above as a rotate right. The literature describes this as one of the "hard cases... when a child is partially contained (in or out = 1). For those cases, construct the two new children, then install them." We'll be using myIn's inPart as a newIn, and making a newOut from myIn's outPart and myOut. The newIn is an OVirtualLoaf, and the newOut is a SplitLoaf.
The SplitLoaf then installs the two new children, adding itself as their OParent, removing itself from its old children as its OParent, and takes the new region [12, +inf) as its mySplit.
And now we've splayed a SplitLoaf. (Say that five times fast!)
The SplitLoaf returns 1 for the number of children within the region, and I've completely lost my bearings. Why exactly were we doing all this splaying, anyway?
TRANSMISSION BEGIN
***ATTENTION: THIS THREAD HAS BEEN RETROACTIVELY DESIGNATED AS ULTRAVIOLET INFORMATION***
IF YOU, OR ONE OF YOUR CLONES, DO NOT HAVE ULTRAVIOLET SECURITY CLEARANCE, PLEASE REPORT TO YOUR NEAREST TERMINATION CENTER FOR SUMMARY EXECUTION.
HAVE A NICE DAY.
Trust The Computer. The Computer is Your Friend.
TRANSMISSION END
*Smashes computer*
Because I've been reading What the Dormouse Said over the last couple of weeks, I've really been thinking a lot about how the culture that we are living in frames the significance of work with computers. A lot of the surface-level hallmarks of what Englebart and Nelson and others came up with have been integrated into what we do today, but filtered through the likes of Gates and Jobs, talented salesmen without the taste (and most likely, the capacity) for the likes of the thought behind human augmentation or Xanadu or the like. So we have the crunchy candy shell without the chocolaty center.
I guess the big question is whether the precommercial concepts of personal computing were the equivalent of the flying cars just around the corner, or whether we're off track. It's good to see the social aspects of the Web rising, but still, the vibe seems different. I wasn't around during the early conceptualizing, though, do I'm a poor judge.
I guess this is a thread to loaf around on, but I just dont get it
|
Senior Software Developer
|
Is a split loaf like a paper clip being used to loosely hold two adjacent sheets of paper together? (Like maps of different areas, clipped together at their border to make a bigger, continuous, map.)
That is :sheet1(paper)::(clip)sheet2:
Part of the paper clip is fastened to one sheet of paper; part of it is fastened to the other.
At the moment I have no understanding of what the splaying is about, so I am unable to help you with your question.
A SplitLoaf represents a branch of the tree in the O (organization) direction. OTrees represent content organized in a particular "space". In the example I've been stepping through, that space is IntegerSpace (basically a number line). A simple region in IntegerSpace would be [0,5) -- starts at 0, goes up to but not including 5.
SplitLoafs represent a split at a particular distinction region. Everything within the distinction goes in the "in" part, and everything outside the distinction goes in the "out" part. Splaying is how you manipulate the trees to ensure this stays true.
This page purports to describe the data structure I'm exploring, but I have my doubts as to whether it really corresponds to the thing I have.
|
Senior Software Developer
|
I guess the big question is whether the precommercial concepts of personal computing were the equivalent of the flying cars just around the corner, or whether we're off track. It's good to see the social aspects of the Web rising, but still, the vibe seems different. I wasn't around during the early conceptualizing, though, do I'm a poor judge.
This is a very good question. Ted Nelson has been arguing strongly for some forty years that we've gone way way way off track. His 70th birthday lecture on Intertwingularity makes that pretty clear. (I also found it entertaining and thought-provoking. How can you not like a lecture that includes a 3-D interactive representation of a Xanalogical space, discussion of language roots and population migration through history, and ends with theories of how the earth might eventually explode?)
In general, I think the reason these visions haven't taken off is that they're really hard problems to solve.
|
Senior Software Developer
|
I've decided I'm not going to complete detailing the spelunking through the machine as it moves a word in a sentence. I finished stepping through the process the other day and it took an hour or so. Eventually I got to the point where I was no longer excited to see a splay in action, because it happens so many times.
I did gain some valuable insights into the workings of the contraption, and have come to the conclusion that all my pictures so far are, well, compelling and pretty, but wrong. In particular, the stuff I colored green and shoved into the upper-left corner and called the "trace canopy" I now believe to be all the stuff that I should have colored blue, for history. (I am not certain about this and may change my mind again after future explorations.) So I'm working on a new diagramming technique. I'd show a picture but after I finished I realized that I might have put the blue stuff upside-down, which is actually kind of important in understanding the beast.
|
Senior Software Developer
|
| 1 person marked this as a favorite. |
The other day, Jason says to me, "Don't be that guy." What guy? Well.... Here I have this enormously complex thing that I don't fully understand, and I'm already thinking of ways that I can make it even more complex and do things that I understand even less. Heh.
The Society of Mind
Twenty years ago, Marvin Minsky published a book called The Society of Mind. Marvin Minsky is a genius artificial intelligence guy at MIT, and the book was his deceptively simple explanation of his theory about how minds work. I say deceptively simple because after reading it, you wonder how come no one's built one of these yet. I mean, he lays out all the parts and how they should interact, complete with little diagrams of the various bits and pieces. Simple!
Well, apparently not so simple. And in fact, the direction AI research has taken over the past two decades has pretty much been the opposite of what he advocated. Rather than building a mind through the interaction of millions of independent agents and isonomes and pronomes and K-lines and frames and whatnot, researchers have focused on things like neural nets and expert systems. He proposes a system built of dozens or hundreds of different kinds of things, of which there are millions and millions of instances, and everyone else apparently has been trying to build intelligent systems by focusing on the interactions of much simpler things.
A couple years ago I re-read The Society of Mind and started toying around with the notion of actually trying to build what Minsky was talking about, for a very specific and narrow purpose: Teaching a robot to make music. I eventually decided that the data structures I was thinking of using (variants on WireHose) were too clumsy to be suited for this purpose and shelved the idea.
A Society of Ents?
Deanster mentioned to me that one of the reasons he thought this contraption might still have some value, even apart from its original intended use to contain all human knowledge, was that AI researchers used K-d trees because you can use them to keep track of related things. I'm pretty sure K-d trees were developed independently of the contraption, but one of the great achievements of its building was a sort of generalized theory of these kinds of things. (I'm not a computer scientist, though I play one on messageboards, so I'm sure I have the details wrong. Sue me.)
And if you think about it, the contraption is all about finding similarities, and knowing when something new has been learned that's similar to something already known, and triggering sensors that make something happen because of that. And that makes up a big part of Minsky's theory.
In particular, take a look at K-lines. K-lines are a model for memory (and other things, like what a mind can do with a memory). It's a conceptual collection of all the agents that were active at a particular time, sort of like writing down everything your brain was doing when you got that "aha!" moment.
An example is, let's say you're repairing a bicycle, and you want to remember all the tools that are useful for fixing a bike. You daub your hand with red paint, so every tool you touch during the repair gets red paint on it. That's a K-line. If you're repairing something else, you use another color of paint. Eventually your tools look like a '60s poster for the Jefferson Airplane, but when you need to repair a bicycle you just pull all the tools that have red on them. That's a K-line.
Now, the pulling out of red-marked tools, and noticing when they might also be marked green or blue, and in what context, is something the contraption does really really well. That's what the canopies are for. In particular, there are several references in the literature about the machine that expressly use the notion of painting things colors and following the painted parts.
For example, extracted from the transcript of this lecture:
The way we do this is we have what we call for historical reasons the Bert canopy, which is sort of the simultaneous projection of all of the H-Trees, or to put it another way, each of these O planes corresponds to a single line down the canopy. So the leaves of the canopy are all the editions and... inside this edition there's a whole bunch of different internal loafs, all of which brownly point at the same canopy crum, so as far as the painting-black operation is concerned, you know that you will always be painting all of these loafs the same way. To put it more accurately, it is always good enough to paint them the same way. What happens by projecting all these guys together, we lose resolution. That we're not getting as fine-grained a filtering as if we had not projected them all completely, because we're taking this good enough, making these guys equivalent to each other. But it's worth it in that we're not losing another factor of log.
And it's worth it because what we're getting in exchange for it, is that to paint this edition black, all we have to do is to paint these canopies crums black. ... Over here, we have a choice point, we're deciding whether to go this way or that way, what we do is take a look at this loaf and follow its brown pointer to the canopy front, and we do the same thing for this other guy over here which has a different projection, so at this choice point, the neighbor has, let's say points at the canopy crum that does not get marked, so what we've done by marking this guy, we've effectively marked all the guys that are pointing at it because while we are engaged in our northward H-Tree walk, we can keep looking over at the canopy to see "do I have the property that I am looking for" and then only taking those paths up the H-Tree that continue to project onto black nodes on the canopy.
So to me this is very interesting. Could you use an Ent to represent agents in the Minsky sense? The most basic approach would be to record sensory data into the Ent. Perhaps, in my music-making robot idea, you create a "MIDI space" which leverages the already computerized definition of musical information.
And maybe you come up with some sort of "agent space" for referring to an agent's relationships with other agents. (Or maybe you don't need to, as the contraption already allows you to build works that contain other editions or works stored in the Ent, and maybe that's good enough for referring to other agents.)
You probably need a simple programming language for describing the plans of how agents interact with each other. Perhaps there's a space you can define for that. I'm told that Ravi (another of the original faithful) actually defined a "function space" at one point, though the team apparently eventually decided that this was too weird even for them. You'd want a space defined for this so agents could record and modify their interactions with other agents over time as new things are learned.
Sensory input would probably be MIDI files. The thing would be like a musical baby, needing to learn concepts like rhythm and melody and harmony, trying to figure out what outputs make it cry for attention and what outputs make it gurgle and smile. I suppose I'd have to be its figurative parent, cooing and encouraging when it makes good sounds, and saying "no, no" when it creates dissonance (at least at first). I imagine communicating with it might be something like R2-D2's beeps and whoops. Eventually it might learn to communicate musically.
Perhaps you could teach it by feeding it a collection of mp3 files, saying, I like these, it feels pleasurable for you to mimic them. Compressed audio files are interesting because they alter the original data to conform to what human ears can hear. I can imagine a musicbot using this to decide what kinds of things are important. Or if you get nicely far down the road, you start showing it pictures and say, This thing you just emitted, it reminds me of this picture. Please make me more things that remind me of that.
Would each agent get its own Ent? Or is there one Ent to hold them all? I don't know the answers to any of these questions, but I like thinking about them.
So... Jason's worried that I'm going to be the guy who enables Skynet, and tells me, "Don't be that guy."
I just want to make a robot that learns to make music.
Is that so wrong?
So... Jason's worried that I'm going to be the guy who enables Skynet, and tells me, "Don't be that guy."
I just want to make a robot that learns to make music.
Is that so wrong?
No, Mr. Fierstein, it isn't wrong or Skynet-like. It's a different 80's film altogether.
"Moles."
"Miles!"
Don't be freaked when your computer starts pumping out Jeff Lynne songs.