(6:03:14 PM) unknown1570: heya
(6:03:18 PM) nofree3DJ: hey
(6:03:28 PM) unknown1570: i'm doing some interesting new stuff with sublims atm
(6:03:37 PM) nofree3DJ: sweet
(6:03:39 PM) nofree3DJ: elaborate
(6:03:53 PM) unknown1570: well, did you ever see the .xs?rc stuff?
(6:04:01 PM) nofree3DJ: uhhh
(6:04:02 PM) nofree3DJ: no?
(6:04:07 PM) unknown1570: not the .xs3d , but the stuff that was called by .xsall
(6:04:12 PM) unknown1570: well okay
(6:04:29 PM) unknown1570: those were general-purpose, non-colour-coded script inputs
(6:04:47 PM) unknown1570: did a lot more processing than the colour coded stuff
(6:05:01 PM) unknown1570: anyway, i've been hacking around with that stuff
(6:05:20 PM) nofree3DJ: what do you mean more processing?
(6:06:44 PM) unknown1570: now i'm catting all my ebooks, manpages, docs (from /usr/docs), and retrieving all my rss feeds, and taking every word, using wordnet to look up all related words (synonyms, antonyms, hypernyms, hyponyms, homonyms, holonyms, etc), and then for each word, looking it up on google, wikipedia, google define, and wikitionary
(6:07:04 PM) unknown1570: i've been interspersing all of that with lots of fortune calls
(6:07:12 PM) unknown1570: expanded my fortune quotes db
(6:07:21 PM) nofree3DJ: fortune calls?
(6:07:28 PM) unknown1570: and i've been catting all my code
(6:07:36 PM) unknown1570: you know of fortune?
(6:07:41 PM) nofree3DJ: forune... 500?
(6:07:49 PM) nofree3DJ: *fortune
(6:07:50 PM) unknown1570: prints a random quote from a large database
(6:07:53 PM) nofree3DJ: ah
(6:08:09 PM) nofree3DJ: so your creating a personal fortune db?
(6:08:14 PM) unknown1570: they took it out of most binary distros due to copyright stuff
(6:08:15 PM) unknown1570: no
(6:08:26 PM) unknown1570: i'm taking all the large fortune dbs and installing them
(6:08:36 PM) nofree3DJ: oh
(6:08:38 PM) unknown1570: and running fortune repeatedly in my sublim feed scripts
(6:08:48 PM) unknown1570: you know, xsublim -program script.sh
(6:08:55 PM) nofree3DJ: yews
(6:08:57 PM) nofree3DJ: *yes
(6:09:26 PM) unknown1570: and often, the fortune quotes are looked up with the wordnet stuff, on wikipedia, google, etc.
(6:10:00 PM) unknown1570: basically, i'm grabbing a ton of data, processing it for any semantically similar data, and looking up all variations
(6:10:13 PM) unknown1570: actually, any semantically related data
(6:10:26 PM) unknown1570: and then, i'm putting all that into sublims
(6:10:27 PM) nofree3DJ: I see
(6:10:31 PM) unknown1570: loads of separate streams
(6:10:36 PM) nofree3DJ: loads?
(6:10:40 PM) unknown1570: tons
(6:10:41 PM) nofree3DJ: like > 10
(6:10:41 PM) unknown1570: lots
(6:10:44 PM) nofree3DJ: > 100??
(6:10:47 PM) unknown1570: 25
(6:10:50 PM) unknown1570: approximately
(6:10:51 PM) nofree3DJ: ok lol
(6:10:54 PM) unknown1570: maybe more like 30
(6:11:16 PM) unknown1570: the new kernel and the new glibc make it possible to do this
(6:11:31 PM) unknown1570: updated the kernel to use the new cfq, new glibc, plus a newer xorg
(6:11:43 PM) unknown1570: speeds things up immensely
(6:11:51 PM) unknown1570: nice -20'd basically everything
(6:11:52 PM) nofree3DJ: cfq?
(6:12:03 PM) unknown1570: cfq is the completely fair scheduler
(6:12:11 PM) unknown1570: it's the new standard scheduler in 2.6
(6:12:25 PM) unknown1570: intro'd it in 2.6.19.x
(6:12:40 PM) nofree3DJ: so it's a nice replacement?
(6:12:41 PM) unknown1570: became the standard in 2.6.21.x
(6:12:44 PM) unknown1570: no
(6:13:09 PM) unknown1570: it simply replaces the current scheduler, which relies on hacky usage heuristics
(6:13:31 PM) nofree3DJ: and isn't the current scheduler nice?
(6:13:56 PM) unknown1570: but yeah, the amount of base ebook text (which would be expanded many times over by wordnet, wikipedia, google, etc.) is 150+ mb
(6:13:58 PM) unknown1570: no
(6:14:11 PM) unknown1570: nice is a program/api call that sets the priority IN the scheduler
(6:14:23 PM) nofree3DJ: ok
(6:14:24 PM) unknown1570: the niceness is synonymous with the timeslice priority
(6:14:26 PM) nofree3DJ: I understand
(6:15:31 PM) unknown1570: cfq has a new way of scheduling by priority (the old scheduler used a combination of a variation of round robin and a kind of automatic renicing based on statistical usage heuristics)
(6:16:08 PM) unknown1570: cfq makes most of the heuristics unnecessary, but some are still in there (which is a godsend, since i don't think i can manually renice my mplayer output every few secs)
(6:16:13 PM) unknown1570: anyway
(6:16:18 PM) nofree3DJ: lol
(6:16:28 PM) unknown1570: a lot of the new ebook data is really interesting
(6:17:26 PM) unknown1570: high level math, philosophy, quantum physics, linguistics, music, audio stuff, cogsci, neuroscience, materials for learning various languages, ai algorithms, signal processing
(6:17:55 PM) unknown1570: it actually works much better, it seems, if the data is meaningful but difficult to understand, and very new
(6:18:09 PM) unknown1570: my level of cryptomnemonics has increased significantly
(6:18:20 PM) nofree3DJ: cryptomnemonics?
(6:18:21 PM) unknown1570: i suddenly know lots of stuff about stuff i've never read about
(6:18:50 PM) unknown1570: cryptomnemonics = unknown memories / mysterious knowledge
(6:18:54 PM) nofree3DJ: ah
(6:19:03 PM) unknown1570: it's what happens when you know something but can't imagine where you learned it
(6:19:35 PM) nofree3DJ: so is this data harvesting automatic? how did you acheive that?
(6:19:36 PM) unknown1570: i also came up with several new units of terminology
(6:19:53 PM) unknown1570: oh... lynx (a commandline-base browser) has a -dump utility
(6:20:14 PM) unknown1570: you can pass a url to lynx -dump and it prints the document, sans markup
(6:20:24 PM) nofree3DJ: sweet
(6:20:27 PM) nofree3DJ: didn't know about that
(6:20:31 PM) unknown1570: so, simply sub variables into the url
(6:20:48 PM) unknown1570: ex.: lynx -dump http://google.com/q=${queryword}
(6:21:09 PM) unknown1570: lynx is really old-school
(6:21:14 PM) nofree3DJ: could you send me your code?
(6:21:20 PM) unknown1570: my sublim scripts?
(6:21:23 PM) unknown1570: sure
(6:21:31 PM) nofree3DJ: the data harvesting script?
(6:21:33 PM) unknown1570: oh
(6:21:36 PM) nofree3DJ: I have the .3x
(6:21:40 PM) unknown1570: that's all built into the sublim scripts
(6:21:43 PM) nofree3DJ: ah
(6:21:45 PM) nofree3DJ: ohhh
(6:21:46 PM) nofree3DJ: I see
(6:21:50 PM) unknown1570: aside from the rss feed reader
(6:21:56 PM) nofree3DJ: saves disk space
(6:21:59 PM) nofree3DJ: lol
(6:22:07 PM) unknown1570: you see, i'm dumping all my rss feeds in there too
(6:22:14 PM) nofree3DJ: I was thinking you had a 10GB db of text documents :P
(6:23:10 PM) unknown1570: nah, i have 150+ mb of text documents, used as a "seed" for the actual main harvesting, as well as some base materials
(6:23:26 PM) nofree3DJ: how can you be sure you won't be repeating data?
(6:25:19 PM) unknown1570: i'm catting all those text documents, the rss feed contents, all my manpages, all the stuff in /usr/doc , and all my code in one script, and that's fed directly into one sublim stream, while another stream is connected to a script that feeds that script, word by word, into the wordnet utility to get all related words, then gets the google search results, the google define results, and the wikipedia results for each related word
(6:25:44 PM) unknown1570: it's pretty cool actually
(6:25:56 PM) nofree3DJ: but what about useless words and articles
(6:27:01 PM) nofree3DJ: and stuff like that....
(6:29:46 PM) nofree3DJ: err..
(6:31:54 PM) unknown1570: doesn't matter really
(6:32:11 PM) unknown1570: that noise doesn't make much of a difference
(6:32:27 PM) unknown1570: it happens fast enough that there's not much of an issue
(6:32:27 PM) nofree3DJ: I see
(6:32:48 PM) nofree3DJ: but wouldn't it be rather simple to remove articles and such, or would that reduce the speed
(6:32:56 PM) unknown1570: remove articles?
(6:33:01 PM) unknown1570: oh
(6:33:36 PM) unknown1570: it'd reduce the speed a bit i guess to parse for useful stuff, but it's not really an issue; it's a waste of effort to do that
(6:33:46 PM) nofree3DJ: I see
(6:33:54 PM) nofree3DJ: can you send me the script?
(6:34:00 PM) unknown1570: sure
(6:34:03 PM) nofree3DJ: thanks
(6:34:09 PM) unknown1570: it's actually a couple of scripts
(6:34:17 PM) nofree3DJ: ok
(6:34:40 PM) unknown1570: you want the bit that harvests catted data, or the bit that cats the data from various sources?
(6:35:00 PM) unknown1570: i can just give you the simple template for grabbing the entries for any word
(6:35:21 PM) nofree3DJ: uhh
(6:35:23 PM) nofree3DJ: both?
(6:35:35 PM) unknown1570: for item in $(cat ~/stories/ebooks/*.txt | ~/markov.lua) ; do
for item2 in $(/usr/local/WordNet-2.0/bin/wn $item -simsv -over -causv -meron -attrn -attra -antsn -antsv -antsa -antsr -coorn -coorv -holon -ssubn -framv -hypen -hypev -hypon -hypov) ; do
fortune
fortune
lynx -dump http://en.wikipedia.org/wiki/${item2}
fortune
fortune
lynx -dump http://google.com/search?q=define:${item2}
fortune
fortune
lynx -dump http://google.com/search?q=${item2}
fortune
done
done
(6:36:17 PM) nofree3DJ: sorry
(6:36:24 PM) nofree3DJ: that formatting came out weird over here
(6:36:25 PM) nofree3DJ: http://pastebin.ca/
(6:36:28 PM) unknown1570: you just substitute the $(cat ~/stories/ebooks/*.txt | ~/markov.lua) with whatever the script is to cat those programs
(6:36:35 PM) unknown1570: formatting came out wierd?
(6:36:41 PM) nofree3DJ: yes..
(6:36:41 PM) unknown1570: it's still readable though, right?
(6:36:54 PM) nofree3DJ: h/o
(6:37:02 PM) nofree3DJ: yea
(6:37:09 PM) unknown1570: the formatting is kind of screwy in the original tbh
(6:37:16 PM) unknown1570: i wrote it really fast :P
(6:37:29 PM) unknown1570: you need the wordnet app though
(6:37:34 PM) nofree3DJ: ok
(6:37:44 PM) unknown1570: which requires downloading the tarball and running make install
(6:37:48 PM) unknown1570: you also need lynx
(6:37:51 PM) unknown1570: and fortune
(6:38:38 PM) unknown1570: fortune serves both as random data and as padding for the google queries, since google doesn't like people grabbing their results many times very fast
(6:39:10 PM) nofree3DJ: why not just wait?
(6:39:25 PM) unknown1570: because then the time to data ratio would be off
(6:39:29 PM) nofree3DJ: ok
(6:39:41 PM) unknown1570: this way, it waits, but no time is wasted because you fill the missing time in with data
(6:39:51 PM) unknown1570: which requires no downloads anyway
(6:40:07 PM) unknown1570: i suggest liberally interspersing fortune into whatever catting script you use anyway
(6:40:26 PM) unknown1570: since most of the fortune quotes contain words prime for looking up anyway
(6:40:50 PM) unknown1570: but i suggest having lots of large text files filled with a lot of stuff you don't understand as well
(6:41:28 PM) unknown1570: but also, remember to feed the catting script directly into one sublim stream, while that mining script is plugged into a different stream
(6:41:32 PM) unknown1570: anyway
(6:41:36 PM) unknown1570: terminology
(6:41:42 PM) unknown1570: i have two terms
(6:42:08 PM) unknown1570: 1) inframation = subliminal information (infra = sub)
(6:43:14 PM) unknown1570: 2) KASEI = Knowledge Amplification by Stimulated Emission of Inf{ra,or}mation (pun on LASER = Light Amplification by Stimulated Emission of Radiation)
(6:43:28 PM) unknown1570: KASEI is my new term for the infornography treatment
(6:43:40 PM) unknown1570: inframation is my new term for sublim content
(6:43:58 PM) unknown1570: or alternately, for data stored in the subconcious, accessible to intuition
(6:45:36 PM) unknown1570: strangely, inframation literally becomes "beneath motion" -- funny, because on the one hand, the data is moving in the subconcious, and on the other hand, when you're sublimming you're usually not moving much :P
(6:46:46 PM) unknown1570: oh, in your choice of stream input, remember to put lots of idiosyncratic/paradoxical stuff, as well as puns and "jake" statements (statements that are inherently confusing because they initially seem to mean one thing, but once the sentence progresses, a different meaning is clear)
(6:47:14 PM) unknown1570: also, double entendres (preferably tame ones) are good -- ambiguous statements, even if there are more than two meanings
(6:47:36 PM) nofree3DJ: tame ones - lol
(6:47:37 PM) unknown1570: and lots of emotionally or intellectually provocative statements
(6:47:47 PM) nofree3DJ: ok but you forgot one thing
(6:47:53 PM) nofree3DJ: I'm not a bash programmer
(6:48:05 PM) nofree3DJ: I don't know how to 'pass data streams' and such
(6:48:36 PM) unknown1570: well, the Van Kaplan effect has the innate capability to affect your personality even when you're not aware of reading the sublim, and we don't want you turning into a pervert
(6:48:38 PM) unknown1570: ah
(6:48:40 PM) unknown1570: just cat them
(6:49:00 PM) nofree3DJ: how do I pass them from script to script
(6:49:03 PM) unknown1570: write a program that cats all the files in a directory
(6:49:36 PM) unknown1570: and then to use that in another script, do $(/path/to/firstscript.sh)
(6:50:02 PM) unknown1570: for example, if i was using markov.sh in my home directory, $(~/markov.sh)
(6:50:39 PM) unknown1570: $(script) substitutes as a string containing all the text output from script
(6:50:56 PM) nofree3DJ: wait
(6:51:02 PM) nofree3DJ: the orignial script you gave me
(6:51:07 PM) unknown1570: i already have the $() thing in that one
(6:51:12 PM) nofree3DJ: that is going to dump it to an output not a variable
(6:51:21 PM) unknown1570: yeah, of course
(6:51:32 PM) nofree3DJ: so that is in the same one as the catting script?
(6:51:48 PM) unknown1570: yeah
(6:51:55 PM) nofree3DJ: then what is the other script
(6:52:02 PM) unknown1570: just write the name of the catting script in one xsublim call
(6:52:24 PM) unknown1570: and the other one (the one i pasted), you put the name of the catting script in the first $() bit
(6:52:29 PM) unknown1570: replacing what's there
(6:52:35 PM) unknown1570: that's the mining script
(6:52:48 PM) unknown1570: and you put the name of that script in another xsublim call
(6:52:54 PM) unknown1570: anyway, i gotta go
(6:52:57 PM) unknown1570: see ya
(6:52:58 PM) nofree3DJ: ok
(6:52:59 PM) nofree3DJ: thanks man
(6:53:00 PM) nofree3DJ: bye