2008-09-12 00:00 <MaZe> as did I
2008-09-12 00:03 <flips> good to do this stuff in public
2008-09-12 00:03 <flips> keeps a useful record, and it's a subtle way of poking at the libxattr guys to fix their packages
2008-09-12 00:03 <flips> 
ACTION wonders who the libxattr guys are

2008-09-12 00:04 <flips> choose one: 1) redhat 2) suse
2008-09-12 00:05 <konrad> http://oss.sgi.com/projects/xfs/
2008-09-12 00:05 <flips> possible
2008-09-12 00:06 <konrad> $ rpm -qf `which getfattr `
2008-09-12 00:06 <konrad> attr-2.4.41-1.fc9.x86_64
2008-09-12 00:06 <flips> it was a suse guy who put xattr+acl support into ext3
2008-09-12 00:06 <konrad> $ rpm -qi attr | grep URL
2008-09-12 00:06 <konrad> URL         : http://oss.sgi.com/projects/xfs/
2008-09-12 00:06 <konrad> ah, ok
2008-09-12 00:06 -!- stargazr5(~gauravstt@59.95.35.250) has joined #tux3
2008-09-12 00:07 <flips> well all the slashdotters seem to agree that the new lucasarts game lacks in the gameplay department
2008-09-12 00:07 <flips> it's fun though
2008-09-12 00:08 <flips> for a few minutes
2008-09-12 00:08 <flips> throwing big chunks of metal at little robots
2008-09-12 00:08 <flips> making lightning come out of your fingers and not do much
2008-09-12 00:10 <MaZe> what's it called?
2008-09-12 00:11 <flips> force unleashed
2008-09-12 00:11 <flips> you play as a sith apprentice
2008-09-12 00:11 <flips> darth's personal waterboy
2008-09-12 00:14 <cdk> flips: tried KOTOR?
2008-09-12 00:14 <cdk> both parts are good
2008-09-12 00:14 <flips> enjoyed kotor
2008-09-12 00:15 <cdk> force unleashed better ?
2008-09-12 00:16 <flips> but I don't want to play any more bioware games
2008-09-12 00:16 <flips> too cookie cutter
2008-09-12 00:16 <flips> even jade empire felt like kotor
2008-09-12 00:16 <flips> and I got spoiled by oblivion
2008-09-12 00:16 <flips> everything you couldn't do in kotor,you could do in oblivion
2008-09-12 00:16 <MaZe> nope. haven't played either
2008-09-12 00:16 <cdk> yeah. oblivion was great.
2008-09-12 00:17 <flips> force unleashed is mildly entertaining
2008-09-12 00:17 <flips> they got some mechanics right, others badly wrong
2008-09-12 00:17 <flips> and it's way linear
2008-09-12 00:18 <flips> I'll probably get force unleashed
2008-09-12 00:18 <flips> but I don't expect much
2008-09-12 00:18 <flips> just filling time until bethesda comes up with something new ;-)
2008-09-12 00:18 <cdk> :) 
2008-09-12 00:19 <cdk> thanks for ur support to the de-duplication idea
2008-09-12 00:19 <cdk> will get back to you when we have something more concrete
2008-09-12 00:19 <flips> welcome, so that's you
2008-09-12 00:20 <flips> sure
2008-09-12 00:20 <flips> deduplication seems to be a big hot button
2008-09-12 00:20 <cdk> yeah...me, stargazr5 , kd and another
2008-09-12 00:20 <flips> for new fs design
2008-09-12 00:20 <flips> ah
2008-09-12 00:20 <flips> were you here for the vfs tour today?
2008-09-12 00:21 <cdk> no....missed it... :( but following it now. 
2008-09-12 00:21 <flips> good
2008-09-12 00:21 <flips> how many years of C has each of you got?
2008-09-12 00:22 <MaZe> look through the logs - there's bound to be some jewels in there
2008-09-12 00:23 <cdk> about 2 and half years
2008-09-12 00:23 <cdk> MaZe: thanks. will do.
2008-09-12 00:24 <flips> and have you done an OS and/or FS course yet?
2008-09-12 00:24 <flips> I think this is for an advanced fs course, right?
2008-09-12 00:24 <flips> 
ACTION reads again

2008-09-12 00:25 <cdk> yes. OS.
2008-09-12 00:26 <MaZe> how much low level experience do you have? C / assembly interfaces?  While it's not really needed, it comes in useful from time to time.
2008-09-12 00:27 <MaZe> [of course, IMHO, assembly is always useful to know... so I may be biased]
2008-09-12 00:27 <flips> I agree
2008-09-12 00:27 <flips> not as a first language
2008-09-12 00:27 <flips> but certainly as one of the first 5
2008-09-12 00:28 <flips> it's been years now since I've written any
2008-09-12 00:28 <flips> if I do write some, it's likely to be for some strange arch like cell spe
2008-09-12 00:28 <MaZe> oh, I've never written true assembly... it's always been inline, often entire procedures, but never entire programs (unless the entire program was a hundred lines or less)
2008-09-12 00:29 <flips> ah, I've written tens of thousands of lines
2008-09-12 00:29 <flips> wallowed in it
2008-09-12 00:29 <cdk> have a bit of experience in assembly...not much..
2008-09-12 00:29 <MaZe> oh, I've written tens of thousands of lines, never all in one piece though
2008-09-12 00:29 <flips> got really good at it, then realized there's people much, much better
2008-09-12 00:30 <stargazr5> this is part of project that we can do on any cs topic
2008-09-12 00:30 <flips> right
2008-09-12 00:30 <MaZe> The biggest pieces I've written were usually either asm-coded bigint adders and the like, or something like a boot sector
2008-09-12 00:30 <flips> just reread your post
2008-09-12 00:30 <flips> I've done stuff llike transcoded knuth's algorithms for infinite precision math from MIX and x86
2008-09-12 00:31 <MaZe> neither of which has a lot of code, but either has huge performance boosts from assembly, or just needs to be in asm
2008-09-12 00:31 <flips> making the carries work out is hard ;-)
2008-09-12 00:31 <flips> MIX to x86 I mean
2008-09-12 00:31 <MaZe> oh, but that sort of stuff is still not pure asm, you write that in C with good macros and inline asm
2008-09-12 00:31 <flips> not me
2008-09-12 00:31 <flips> pure
2008-09-12 00:31 <MaZe> there is almost never a reason to write pure asm
2008-09-12 00:32 <flips> sure, when the OS isn't linux
2008-09-12 00:32 <flips> and the compiler isn't gcc
2008-09-12 00:32 <flips> gcc asm syntax blows by the way ;-)
2008-09-12 00:32 <MaZe> true, if compiler != gcc, then hang yourself
2008-09-12 00:32 <MaZe> blows as in bad? or in good?
2008-09-12 00:32 <flips> bad
2008-09-12 00:32 <flips> sucks chunks
2008-09-12 00:32 <MaZe> the syntax is bad, but it is extremely powerful
2008-09-12 00:33 <MaZe> although it takes quite some getting used to
2008-09-12 00:33 <flips> I know, so why not have the syntax be good and be extremely powerful?
2008-09-12 00:33 <MaZe> hehe
2008-09-12 00:33 <MaZe> yeah, well... that's like
2008-09-12 00:33 <MaZe> C
2008-09-12 00:33 <MaZe> the syntax for C also blows
2008-09-12 00:33 <flips> kinda yes
2008-09-12 00:33 <flips> at&t asm blows orders worse
2008-09-12 00:33 <flips> I think that's where it came from
2008-09-12 00:34 <flips> well
2008-09-12 00:34 <flips> let's not scare the visitors ;-)
2008-09-12 00:34 <flips> cranky old hacks
2008-09-12 00:35 <MaZe> a (*b(c d, e (*f)(g)))(h);
2008-09-12 00:35 <MaZe> what kind of syntax is that?
2008-09-12 00:35 <stargazr5> next tux3 university?
2008-09-12 00:36 <MaZe> tue at 8 pm pacific
2008-09-12 00:36 <flips> tuesday 8 pm
2008-09-12 00:36 <flips> right, I tend to forget that timezone
2008-09-12 00:36 <cdk> will be there this time.
2008-09-12 00:36 <stargazr5> ok
2008-09-12 00:36 <flips> like the world revolves around silly valley
2008-09-12 00:36 <konrad> :)
2008-09-12 00:36 <stargazr5> :)
2008-09-12 00:36 <MaZe> doesn't it?
2008-09-12 00:36 <flips> not sure
2008-09-12 00:36 <flips> I didn't use to think so ;)
2008-09-12 00:37 <MaZe> I have some stories about silicon valley and time zones that I can't share ;-(
2008-09-12 00:37 <flips> nice to know there's a reason to get you drunk
2008-09-12 00:38 <flips> speaking of which
2008-09-12 00:38 <flips> a sake would help get this post written
2008-09-12 00:38 <flips> or should I just hack
2008-09-12 00:38 <flips> hmm
2008-09-12 00:38 <flips> I mean, with the sake in hand of course
2008-09-12 00:39 <MaZe> [btw, that declaration above, that's valid C - that's the declaration of signal from the std C library, where a,e=void b=signal c,g,h=int d=sig f=func
2008-09-12 00:39 <cdk> taking abt timezones...its luch time here...i am off..
2008-09-12 00:40 <flips> wow
2008-09-12 00:40 <stargazr5> me too
2008-09-12 00:41 <flips> see you cdk, stargazr5
2008-09-12 00:42 <flips> you know I can read that without thinking?
2008-09-12 00:42 <flips> that's scary
2008-09-12 00:42 <flips> also used to to hex multiply/divide in my head at the most geekiest
2008-09-12 00:42 <flips> still can do it, more slowly
2008-09-12 00:50 <MaZe> the C syntax above? without thinking? really? that is scary
2008-09-12 00:50 <MaZe> that is like the ugliest part of C...
2008-09-12 00:51 <flips> arguably
2008-09-12 00:51 <flips> personally, I think const is
2008-09-12 00:51 <flips> dreamed up by a sadist
2008-09-12 00:51 <MaZe> oh, const is relatively simple to parse though
2008-09-12 00:51 <MaZe> especially if you write it so 'char const *' instead of 'const char *'
2008-09-12 00:51 <flips> but a devilishly effective makework project
2008-09-12 00:52 <MaZe> and then read from the back, since C is mostly read from the back/center anyway
2008-09-12 00:52 <flips> bottom up reading is a powerful organizing force
2008-09-12 00:52 <MaZe> I so much prefer Pascal syntax for type definitions
2008-09-12 00:52 <MaZe> you just read it left to right
2008-09-12 00:52 <flips> me too
2008-09-12 00:53 <flips> but pascal as a whole is just plain irritating
2008-09-12 00:53 <MaZe> if the things a pointer it says so in the first character
2008-09-12 00:53 <flips> I am also offended by ==
2008-09-12 00:53 <MaZe> irritating? a little long-winded, true, but so frickin' easy to understand
2008-09-12 00:53 <flips> and friends
2008-09-12 00:53 <MaZe> yes, I prefer the pascal := and = as opposed to = and ==
2008-09-12 00:54 <MaZe> I don't mind != nor <> -  doesn't really matter to me
2008-09-12 00:54 <flips> the most pleasant language I've worked in is pick basic
2008-09-12 00:54 <flips> as modified by a friend of mine
2008-09-12 00:54 <MaZe> not aware of that - does that differ from basic?
2008-09-12 00:54 <flips> most of the stupidities gone
2008-09-12 00:54 <flips> and missing stuffyou need in, like structuring primitives
2008-09-12 00:54 <MaZe> [are you aware modern oo pascal like delphi, like freepascal, is 32bit and has object oriented programming, operator overloading, function overloading, etc...]
2008-09-12 00:55 <flips> it's very different from basic
2008-09-12 00:55 <flips> totally not anything like msft basic
2008-09-12 00:55 <MaZe> h
2008-09-12 00:55 <MaZe> ehrm, ah
2008-09-12 00:55 <flips> I haven't used the latest borland stuff, no
2008-09-12 00:55 <flips> msft liked it enough to headhunt the guy as I recall
2008-09-12 00:55 <flips> and we got c# :p
2008-09-12 00:56 <flips> another flavor of C-that-blows
2008-09-12 00:56 <konrad> C# is just a flavour of java
2008-09-12 00:56 <flips> with the worst of C added in
2008-09-12 00:58 <MaZe> I want a melding of pascal (type declaration syntax, ease of reading code), Java (generics, interfaces [extended]), gnu-ism (inline asm power), C (low level control), C++ (some of the OO, dropping multiple inheritance), not sure what to do with some things [exceptions]
2008-09-12 00:59 <flips> let me know when you have code to try
2008-09-12 00:59 <MaZe> make it a very strongly typed language, drop most of the legacy crap, support useful UI candy (type in constants in any base, etc...)
2008-09-12 01:00 <flips> don't forget to make it interactive
2008-09-12 01:00 <flips> and managed
2008-09-12 01:00 <flips> and semicolons optional
2008-09-12 01:00 <MaZe> being able to recompile program blocks and replace them on the fly - yes I've wanted that ;-)
2008-09-12 01:00 <MaZe> managed?
2008-09-12 01:00 <flips> likewise parens, including fn call parens
2008-09-12 01:00 <MaZe> a function needs parens
2008-09-12 01:00 <flips> managed = can't segfault
2008-09-12 01:00 <MaZe> a procedure doesn't
2008-09-12 01:01 <MaZe> what does that mean?
2008-09-12 01:01 <flips> can't segfault?
2008-09-12 01:01 <MaZe> yes.
2008-09-12 01:01 <flips> lisp can't segfault in principle
2008-09-12 01:01 <flips> java can't either
2008-09-12 01:01 <MaZe> oh, but then it's not low-level
2008-09-12 01:01 <konrad> I'm really concerned as to why 65% of my memory is in use (not including cache)
2008-09-12 01:01 <flips> in principle.
2008-09-12 01:01 <flips> konrad, in what context?
2008-09-12 01:01 <MaZe> the above would be a language you could write a kernel in
2008-09-12 01:02 <flips> konrad, running mozilla?
2008-09-12 01:02 <flips> oh
2008-09-12 01:02 <konrad> flips: yeah, but that's only eating 600M or something
2008-09-12 01:02 <konrad> I have 6 gigs
2008-09-12 01:02 <MaZe> I've got 45% in programs on a 4 g machine
2008-09-12 01:02 <flips> ncie
2008-09-12 01:02 <flips> nice
2008-09-12 01:02 <MaZe> 50% cache
2008-09-12 01:02 <konrad> yeah. I'm concerned.
2008-09-12 01:02 <flips> need to get yourself a memory map
2008-09-12 01:02 <flips> from proc
2008-09-12 01:02 <flips> there must be a tool
2008-09-12 01:03 <MaZe> (and in my case that probably is a gig and a half of firefox3)
2008-09-12 01:03 <MaZe> wicked
2008-09-12 01:03 <MaZe> 13457 maze      20   0  103m  12m 9.9m S  0.0  0.3   0:29.40 gnome-power-man  
2008-09-12 01:04 <MaZe> 103M!
2008-09-12 01:04 <konrad>  3409 root      20   0 1454m 1.1g  28m S  5.6 18.0 531:18.61 Xorg
2008-09-12 01:04 <flips> just say gno to gnome
2008-09-12 01:04 <konrad> but that's still insubstantial relative to 6
2008-09-12 01:05 <konrad> kmail's using 500M
2008-09-12 01:05 <flips> what do you get from cat /proc/meminfo?
2008-09-12 01:05 <flips> pastie maybe?
2008-09-12 01:06 <konrad> http://pastie.caboo.se/271078
2008-09-12 01:10 <flips> .6 gig of buffers, woof
2008-09-12 01:11 <flips> A gig into swap
2008-09-12 01:11 <flips> that's braindamage
2008-09-12 01:11 <flips> something is using 2.7 gig of straight memory
2008-09-12 01:11 <flips> should not be hard to find
2008-09-12 01:11 <flips> well
2008-09-12 01:11 <konrad> X leaks often
2008-09-12 01:11 <flips> if you don't see it in the processes then yes, that is worrisome
2008-09-12 01:12 <flips> but you'd see the X usage even when it's leaking
2008-09-12 01:12 <flips> you know Shift-M with top?
2008-09-12 01:12 <flips> I think that's the one
2008-09-12 01:12 <flips> shows your proces in rss order
2008-09-12 01:12 <flips> or is it total vm size
2008-09-12 01:12 <flips> one of those
2008-09-12 01:12 <flips> vm size I think
2008-09-12 01:13 <konrad> in order, Xorg, firefox, nautilus, gnome-panel, kmail, pidgin, gnome-terminal
2008-09-12 01:13 <konrad> and some others
2008-09-12 01:13 <flips> VmallocTotal: 34359738367 kB
2008-09-12 01:13 <flips> that has to be broken
2008-09-12 01:14 <flips> VmallocChunk: 34359675895 kB
2008-09-12 01:14 <flips> haven't spent much time crawling in vm lately
2008-09-12 01:14 <flips> you might want to go onto #mm on this server
2008-09-12 01:15 <flips> and complain about that vmalloctotal
2008-09-12 01:15 <konrad> it's late enough that I can't be arsed
2008-09-12 01:15 <flips> works for me
2008-09-12 01:15 <konrad> alright, I got back a significant chunk of it by ditching firefox
2008-09-12 01:16 <konrad> down to 51% used
2008-09-12 01:16 <flips> still
2008-09-12 01:17 <flips> check your anon
2008-09-12 01:17 <flips> I don't think firefox was using that
2008-09-12 01:18 <konrad> 600M of it went away after closing firefox
2008-09-12 01:18 <flips> leaving 2.1 gig in anon?
2008-09-12 01:18 <flips> that's broken
2008-09-12 01:18 <konrad> yes
2008-09-12 01:19 <flips> check top
2008-09-12 01:19 <konrad> for what?
2008-09-12 01:19 <flips> shift-M
2008-09-12 01:19 <flips> look at virtual size
2008-09-12 01:19 <flips> and rss
2008-09-12 01:19 <flips> fes
2008-09-12 01:19 <flips> res
2008-09-12 01:20 <konrad> xorg has 1471m virt 1.1g res, nautilus 767m virt 212m res, kmail 536m virt 146m res
2008-09-12 01:20 <konrad> top 3
2008-09-12 01:20 <flips> fsking pigs
2008-09-12 01:20 <flips> wtf is X doing?
2008-09-12 01:21 <flips> nautilus...
2008-09-12 01:21 <flips> :p
2008-09-12 01:21 <konrad> shh :)
2008-09-12 01:21 <flips> I'd say you'
2008-09-12 01:21 <flips> I'd say you've got a simple case of out of control X plus 4 x bloatware
2008-09-12 01:21 <konrad> sounds about right
2008-09-12 01:22 <flips> I'd call the X part a bug
2008-09-12 01:22 <flips> the other is just sloth
2008-09-12 01:22 <konrad> it leaks like crazy
2008-09-12 01:22 <flips> write nasty emails to xorg
2008-09-12 01:22 <flips> tell them you want your money back
2008-09-12 01:23 <flips> kernel is unsing an unconscionable amount of buffers
2008-09-12 01:23 <flips> metadata is supposed to be small.  Buffers -> metadata
2008-09-12 01:23 <flips> you can go complain on #mm
2008-09-12 01:23 <flips> tell peterz to do something ;-)
2008-09-12 01:23 <konrad> that might be a result of encrypted harddrive
2008-09-12 01:24 <flips> probably
2008-09-12 01:24 <flips> well
2008-09-12 01:24 <flips> dodgy encryption layer
2008-09-12 01:24 <konrad> right
2008-09-12 01:25 <flips> what's the encryption method, craptoloop or dm-crapt?
2008-09-12 01:25 <konrad> dm-crypt
2008-09-12 01:25 <flips> complain on the dm-devel list
2008-09-12 01:25 <flips> there
2008-09-12 01:25 <flips> got it all sorted ;-)
2008-09-12 01:25 <konrad> :)
2008-09-12 01:49 <flips> incremental refcount block update cost <- I love english
2008-09-12 01:49 <flips> don't you maze?
2008-09-12 01:51 <flips> all that stuff in the refcount post was actually designed during the skate today
2008-09-12 01:51 <flips> the same skate the resulting in "rollerbladers are allowed" inthe sk8board park
2008-09-12 01:52 <flips> so it was a good skate, all things considered
2008-09-12 01:52 <flips> g'night
2008-09-12 01:57 <konrad> "Meanwhile, Rockbox has performed a valuable service for Debian developers who would otherwise have to struggle to find a project with longer release cycles than their own. " hah
2008-09-12 01:59 <flips> :-)
2008-09-12 01:59 <flips> what is rockbox?
2008-09-12 02:03 <konrad> different firmware for your ipod
2008-09-12 02:03 <konrad> or other 'mp3 player'-class devices
2008-09-12 02:04 <konrad> (with additional functionality in mind)
2008-09-12 02:09 <bh> 
ACTION talks about b-tree parallelization with flips

2008-09-12 02:09 <bh> I was thinking about how something like RCU could be integrated into a b-tree. I don't know the specifics of a b-tree per se other than it's a tree that's flatter and better suited for storage
2008-09-12 02:11 <bh> flips: so I was thinking about per inode processing if we decided to parallelize it on that bassis
2008-09-12 02:11 <flips> we don't have rcu in userspace
2008-09-12 02:11 <flips> but we need locking in userspace
2008-09-12 02:11 <flips> also: rcu has some scary artifacts
2008-09-12 02:11 <bh> file locking would have to be done on a per inode basis
2008-09-12 02:12 <flips> I think it's a case of, do some decent spinlock + mutex work, then try rcu
2008-09-12 02:12 <flips> not rcu first
2008-09-12 02:12 <flips> rcu is wierd when it goes wrong
2008-09-12 02:12 <bh> so that code would to have to be split up in that manner if you do it that way
2008-09-12 02:12 <bh> yeah, I know
2008-09-12 02:12 <flips> so are spinlocks etc, but the weirdness is a lot easier to grasp
2008-09-12 02:12 <bh> you have to validae the data and stuff before using it
2008-09-12 02:12 <bh> making sure that it's not stale
2008-09-12 02:12 <flips> per inode is too coarse
2008-09-12 02:13 <bh> across quiescence periods
2008-09-12 02:13 <bh> well, what then ?
2008-09-12 02:13 <flips> simple spinlocks and mutexes
2008-09-12 02:13 <flips> decide how many
2008-09-12 02:13 <flips> what order acquired
2008-09-12 02:13 <flips> for how long
2008-09-12 02:13 <flips> what granularity of data protected
2008-09-12 02:14 <flips> estimate contention
2008-09-12 02:14 <flips> decide where rw is appropriate
2008-09-12 02:14 <flips> where trylock works
2008-09-12 02:14 <flips> then think about per cpu
2008-09-12 02:14 <bh> problem here is that rwlocks suck badly since they still depend on an atomic operation, limited scalability which is why some kind of per CPU-ism is useful
2008-09-12 02:14 <flips> at first, pretend bouncing costs nothing
2008-09-12 02:15 <bh> ok
2008-09-12 02:15 <flips> when it's working reliably and bouncing starts to show in the profile (because everything else is so fast) then take anti-bounce measures
2008-09-12 02:15 <bh> let think, how do we want to break this up ?
2008-09-12 02:15 <bh> what are we protecting and what circumstances ?
2008-09-12 02:16 <flips> there should be any number of concurrent readers and writes allowed in one file inode at the same time
2008-09-12 02:16 <flips> same with the inode table
2008-09-12 02:16 <flips> they will be partitioned by subtree
2008-09-12 02:16 <flips> higher levels of the tree can have rwlocks
2008-09-12 02:17 <flips> I think
2008-09-12 02:17 <flips> low level, simple mutex or spinlock is better
2008-09-12 02:17 <bh> what parts of the subtree ?
2008-09-12 02:17 <bh> =and how are they realted to the inode itself ?
2008-09-12 02:17 <bh> and how are they related to the inode itself ?
2008-09-12 02:17 <flips> a file index tree descends from the inode table
2008-09-12 02:18 <flips> think 50 petabyte file
2008-09-12 02:18 -!- kbingham(~kbingham@193.132.141.186) has joined #tux3
2008-09-12 02:18 <bh> well, it depends on what you're guarding, we have to define the relationship first
2008-09-12 02:18 <flips> to get the sense of how many read/writes can be in it at the same time
2008-09-12 02:18 <bh> let's start from the beginning
2008-09-12 02:18 <bh> what happens on a file open ?
2008-09-12 02:18 <flips> guarding changes to the index leaf nodes, which is to say, the block pointers, and later extents
2008-09-12 02:18 <bh> and define the common operations
2008-09-12 02:18 <bh> open, read, write, close
2008-09-12 02:19 <flips> on file open we first look in the directoy file
2008-09-12 02:19 <flips> find the inode number
2008-09-12 02:19 <flips> then probe into the inode table
2008-09-12 02:19 <bh> that's a flat file, right ?
2008-09-12 02:19 <flips> find the inode table block, and the inode in it
2008-09-12 02:19 <flips> the directory?
2008-09-12 02:19 <flips> currently flat
2008-09-12 02:19 <bh> diretory file
2008-09-12 02:19 <bh> directory file
2008-09-12 02:19 <flips> later will have a btree mapped into the flat file
2008-09-12 02:19 <flips> has its own locking considerations
2008-09-12 02:19 <bh> I'm assuming that's it's a specific inode on the file system
2008-09-12 02:20 <flips> what is?
2008-09-12 02:20 <bh> we have two structures so far right ?
2008-09-12 02:20 <bh> 1) directory map file
2008-09-12 02:20 <bh> 2) b-tree
2008-09-12 02:20 <flips> not quite like that
2008-09-12 02:20 <flips> tux3 is a two level btree structure
2008-09-12 02:20 <flips> top level btree is the inode table
2008-09-12 02:21 <flips> from the inode table descend some large number of file index btrees
2008-09-12 02:21 <flips> a directory is the leaves of one of those btrees
2008-09-12 02:21 <flips> that is, the data blocks
2008-09-12 02:21 <flips> the leaves of a file index btree actually contain pointers to data blocks
2008-09-12 02:22 <flips> so we go probing around in some directory, taking the same locks as we would for any file
2008-09-12 02:22 <bh> 
ACTION reads

2008-09-12 02:22 <flips> that is, locking various levels of the index btree of the directory file
2008-09-12 02:23 <flips> once we find a data block we read it into the page cache and drop our locks
2008-09-12 02:23 <flips> maybe not all of them, maybe just up to some level
2008-09-12 02:23 <flips> well
2008-09-12 02:24 <flips> that is a little tricky, because the linux generic_file_read etc functions don't work that way
2008-09-12 02:24 <flips> they generally cause the filesystem to walk its index tree over and over again, for each block
2008-09-12 02:24 <flips> sucks
2008-09-12 02:24 <bh> 
ACTION is a bit confused

2008-09-12 02:24 <bh> 
ACTION thinks

2008-09-12 02:24 <flips> we don't need to worry about that
2008-09-12 02:25 <flips> for the moment we only need to be able to dive down into the btree and find a pointer to some data block
2008-09-12 02:25 <flips> see inode.c
2008-09-12 02:25 <flips> "filemap_blockio"
2008-09-12 02:26 <flips> most of the work is done by "probe"
2008-09-12 02:26 <flips> probe is where most of the locking action will happen
2008-09-12 02:27 <bh> so a file is a b-tree ?
2008-09-12 02:27 <bh> which is lower in level to the inode b-tree ?
2008-09-12 02:27 <bh> that's the relationship ? correct ?
2008-09-12 02:27 <flips> http://tux3.org/tux3?f=6ea2692d2839;file=user/test/btree.c
2008-09-12 02:27 <flips> see probe in there
2008-09-12 02:27 <flips> a file is _indexed_ by a btree
2008-09-12 02:27 <bh> that's in the lower level right ?
2008-09-12 02:28 <bh> 
ACTION looks

2008-09-12 02:28 <flips> a file lives in data blocks, that are pointed to by pointers that live in the leaves of a btree, called a data index btree
2008-09-12 02:28 <flips> the leavesof that btree are called dleaves
2008-09-12 02:28 <flips> see dleaf.c
2008-09-12 02:29 <flips> the situation with dleaf.c is pretty simple
2008-09-12 02:29 <flips> we can protext an entire dleaf as one logical entitity
2008-09-12 02:30 <flips> that covers about 500 file data blocks
2008-09-12 02:30 <flips> which is an ok granularity
2008-09-12 02:30 <bh> top level b-tree right ?
2008-09-12 02:30 <flips> what top level btree?
2008-09-12 02:30 <flips> a dtree is a second level btree
2008-09-12 02:30 <bh> you have an inode b-tree and a data index b-tree, correct ?
2008-09-12 02:30 <flips> the top level btree is the  inode table
2008-09-12 02:30 <flips> right
2008-09-12 02:30 <bh> I'm just trying to understand the terminology here
2008-09-12 02:30 <bh> ok, good
2008-09-12 02:30 <bh> that's what I though
2008-09-12 02:30 <bh> thought
2008-09-12 02:30 <flips> itree vs dtree
2008-09-12 02:31 <bh> good
2008-09-12 02:31 <bh> yeah, good terminology
2008-09-12 02:31 <flips> thanks
2008-09-12 02:31 <bh> itree->dtree
2008-09-12 02:31 <flips> right
2008-09-12 02:31 <bh> terminology is important
2008-09-12 02:31 <flips> agreed
2008-09-12 02:31 <bh> it's what I figured you said in the first place, but I had to be sure
2008-09-12 02:31 <bh> right, protect a dtree entirely with a lock
2008-09-12 02:31 <flips> http://kerneltrap.org/Linux/Tux3_Hierarchical_Structure
2008-09-12 02:32 <flips> some of this is wrong now
2008-09-12 02:32 <bh> 
ACTION reads

2008-09-12 02:32 <flips> dropped the volume table, moved the free map inside the itree
2008-09-12 02:32 <bh> update it before the next tux3 university
2008-09-12 02:32 <flips> as a normal file
2008-09-12 02:32 <flips> that's hard
2008-09-12 02:32 <flips> that's on somebody else's site
2008-09-12 02:32 <flips> but I can post something on tux3.org
2008-09-12 02:32 <bh> 
ACTION really appreciates the help in learning this from flips

2008-09-12 02:33 <bh> do you have an allocation maps that's shared 
2008-09-12 02:33 <bh> ?
2008-09-12 02:33 <flips> at least you can see the inode table / data index table relationship there
2008-09-12 02:33 <flips> but it is obscured by the volume table, which I determined to be useless
2008-09-12 02:33 <bh> that's a potentially huge problem for contention with regards to the allocator
2008-09-12 02:33 <flips> allocation map?
2008-09-12 02:34 <flips> there is an allocation bitmpa
2008-09-12 02:34 <bh> block allocation map
2008-09-12 02:34 <flips> which is a normal file
2008-09-12 02:34 <bh> well, how do you modify it, say, under heavy delete or data creation pressure ?
2008-09-12 02:34 <flips> sb->bitmap in inode.c
2008-09-12 02:34 <bh> doesn't it need a lock around it ?
2008-09-12 02:34 <flips> currently there is no locking
2008-09-12 02:34 <flips> or concurrency
2008-09-12 02:34 <flips> soon
2008-09-12 02:35 <bh> well, doesn't it need it ?
2008-09-12 02:35 <flips> but it is just a normal file
2008-09-12 02:35 <flips> lock it with the same granularity
2008-09-12 02:35 <bh> there's a lot of activity there so I expect it to be heavily hit
2008-09-12 02:35 <flips> sure
2008-09-12 02:35 <bh> same granularity as what ?
2008-09-12 02:35 <flips> other files too
2008-09-12 02:35 <flips> but!
2008-09-12 02:35 <flips> there is a difference with tux3
2008-09-12 02:35 <bh> ok
2008-09-12 02:35 <flips> tux3 has this way of logging changes to the bitmaps
2008-09-12 02:35 <flips> it doesn't have to lock, write block, wait
2008-09-12 02:35 <bh> ok
2008-09-12 02:35 <flips> that kind of thing
2008-09-12 02:35 <bh> oh nice
2008-09-12 02:36 <flips> so locks on the bitmap are just page cache locks
2008-09-12 02:36 <bh> deltas to the allocation map are just appended
2008-09-12 02:36 <flips> that is, most like actually locking pages when we get to kernel
2008-09-12 02:36 <flips> or we could lock buffers
2008-09-12 02:36 <flips> locking pages is a little faster
2008-09-12 02:36 <bh> what about during concurrent access against an online checker that needs to know about all of the appended logs ?
2008-09-12 02:36 <flips> yes, deltas to the allocation map are just logged
2008-09-12 02:37 <flips> and every now and then we pour a bunch of them into the allocation map and write it out
2008-09-12 02:37 <bh> differ those checks until the log has been commit to the disk and then restart it ?
2008-09-12 02:37 <bh> committed
2008-09-12 02:37 <flips> the allocation map always has the most recent version of the allcoation
2008-09-12 02:37 <flips> in buffers
2008-09-12 02:37 <flips> in memory
2008-09-12 02:37 <bh> because, say, you want to verify if data blocks that some indirect mapping is pointing is allocated or not
2008-09-12 02:37 <flips> so an online check, ah, needs to check the disk image
2008-09-12 02:37 <flips> not the cached image
2008-09-12 02:37 <flips> right?
2008-09-12 02:38 <flips> pretty hard to do otheriwse
2008-09-12 02:38 <flips> anway, that's not the immediate problem
2008-09-12 02:38 <flips> the immediate problem is just tohave fast, concurrent access to everything
2008-09-12 02:38 <bh> if it's not and a log is being committed, we should delay it until that log has been committed ?
2008-09-12 02:38 <bh> just thinking out loud
2008-09-12 02:38 <flips> if what is not?
2008-09-12 02:38 <bh> what ? the log ?
2008-09-12 02:39 <bh> the log itself
2008-09-12 02:39 <flips> "if it's not"you said
2008-09-12 02:39 <flips> don't know what "it" is
2008-09-12 02:39 <bh> is there a scenario where the online checking of that portion of the disk and ...on that'll never happen
2008-09-12 02:39 <bh> because of the atomic commit
2008-09-12 02:39 <bh> it's should be consistent at that point from previous commits
2008-09-12 02:40 <bh> we don't really need to check logs that are being committed to disk and wait for them to complete
2008-09-12 02:40 <bh> or do we ?
2008-09-12 02:40 <flips> we do
2008-09-12 02:40 <flips> because the logs form a promise of what the "real" disk image "should" look like
2008-09-12 02:40 <bh> yeah, well then we have to lock them down or something like that
2008-09-12 02:40 <flips> so we need to take it into account during checking
2008-09-12 02:40 <flips> but checking is far in the future
2008-09-12 02:41 <bh> under, say a rwlock lock, reader side
2008-09-12 02:41 <flips> at least 3 months
2008-09-12 02:41 <flips> probably 4
2008-09-12 02:41 <bh> ok
2008-09-12 02:41 <flips> worth thinking about
2008-09-12 02:41 <flips> let's do beer on it
2008-09-12 02:41 <flips> I'll think about it on my next skate, if refcounting is done ;-)
2008-09-12 02:42 <flips> going back to the bitmap
2008-09-12 02:42 <flips> so, at least each bit has to be protected
2008-09-12 02:42 <flips> we do scan, find, change
2008-09-12 02:42 <flips> and that scan/find/change to allocate a block has to be under a spinlock
2008-09-12 02:43 <bh> flips: am I being helpful or not ?
2008-09-12 02:43 <flips> or in userspace, under a pthread mutex
2008-09-12 02:43 <bh> or saying stupid irrelevant things ? just checking
2008-09-12 02:43 <flips> of course
2008-09-12 02:43 <bh> ok
2008-09-12 02:43 <flips> I haven't been required to be precise about this up till now
2008-09-12 02:43 <flips> or deal with somebody who had written nontrivial locking
2008-09-12 02:43 <bh> we'll I hope I'm helping
2008-09-12 02:43 <flips> yep
2008-09-12 02:43 <flips> anyway, the allocation bitmap is a good place to start
2008-09-12 02:44 <flips> because there is a pretty simple situation there
2008-09-12 02:44 <bh> I think you can definitely isolate a dtree using an individual dtree lock
2008-09-12 02:44 <flips> once you know your bitmap block isn't going away
2008-09-12 02:44 <flips> well, one lock per dtree is way too crude
2008-09-12 02:44 <bh> that's good, but you have to think about the upward relationship between than than itree
2008-09-12 02:44 <flips> actually you don't
2008-09-12 02:44 <flips> you can treat it as individual blocks
2008-09-12 02:44 <bh> is the lock against the dtree sufficient to protect the link in the itree pointing to it ?
2008-09-12 02:45 <bh> stuff like that
2008-09-12 02:45 <flips> you lock your way down through the btree until you get to the datablock, lock the data block and let everything else go
2008-09-12 02:45 <bh> ok, let's define what a read would look like through that.
2008-09-12 02:45 <flips> ok, right
2008-09-12 02:45 <bh> you look up the inode in the itree
2008-09-12 02:45 <bh> you get it
2008-09-12 02:45 <bh> what next ?
2008-09-12 02:45 <bh> it points to a dtree
2008-09-12 02:45 <flips> well that's a good point
2008-09-12 02:45 <flips> so you want to delete a data block
2008-09-12 02:45 <flips> that means you have to lock the data block
2008-09-12 02:46 <flips> so I think that's clear, right?
2008-09-12 02:46 <bh> yes
2008-09-12 02:46 <flips> that means, the read has to be off it
2008-09-12 02:46 <flips> reader
2008-09-12 02:46 <bh> what do you lock then ? the dtree or some part of the dtree ?
2008-09-12 02:46 <flips> you lock the block
2008-09-12 02:46 <bh> how would region locking look like ?
2008-09-12 02:46 <flips> region locking looks like locking a subtree node
2008-09-12 02:46 <flips> then you have to wait for _every_ other lock to go away
2008-09-12 02:47 <flips> not a good idea
2008-09-12 02:47 <flips> why do you want to lock a region?
2008-09-12 02:48 <bh> posix semantics or something like that
2008-09-12 02:48 <flips> totally different locking level
2008-09-12 02:48 <bh> can't you lock a range in the file under posix ?
2008-09-12 02:48 <flips> waaay up in the vfs
2008-09-12 02:48 <flips> layered, independent
2008-09-12 02:49 <bh> also, the linux posix locking code blows
2008-09-12 02:49 <bh> coarse grained as hell
2008-09-12 02:49 <flips> true, but you can bypass it
2008-09-12 02:49 <bh> single fucking lock
2008-09-12 02:49 <flips> yup, and a linear list
2008-09-12 02:49 <flips> blows
2008-09-12 02:49 <flips> but you still do it at the same level
2008-09-12 02:49 <flips> that's not our concern now
2008-09-12 02:50 <flips> or maybe I'm just stuck in the wrong mindset
2008-09-12 02:50 <flips> could be
2008-09-12 02:50 <flips> anyway, at least we can let that suck exaclty as it always has
2008-09-12 02:50 <flips> we won't lose a benchmark showdown for that reason
2008-09-12 02:50 <flips> who uses posix locks anyway? ;)
2008-09-12 02:52 <flips> there is a case where you want to lock a region
2008-09-12 02:52 <flips> cluster fs
2008-09-12 02:52 <flips> but that's not us
2008-09-12 02:52 <flips> yet
2008-09-12 02:53 <bh> ok
2008-09-12 02:53 <bh> let's continue
2008-09-12 02:53 <bh> how do you lock the data block ?
2008-09-12 02:53 <flips> I guess for tux3 we can think of a single block as our unit of locking
2008-09-12 02:53 <bh> this is for a read remember...
2008-09-12 02:53 <flips> in kernel?
2008-09-12 02:53 <flips> take the block lock
2008-09-12 02:53 <flips> it's a bitspin lock as I recall
2008-09-12 02:54 <flips> same with the page lock
2008-09-12 02:54 <flips> it's fast enough for this purpose
2008-09-12 02:54 <bh> we'll its all something to think about
2008-09-12 02:54 <flips> in userspace
2008-09-12 02:54 <flips> pthread mutex, we will put one in each buffer
2008-09-12 02:54 <bh> that's pretty nasty
2008-09-12 02:54 <flips> so you lock the mutex in the buffer
2008-09-12 02:54 <flips> because?
2008-09-12 02:54 <bh> how big of a file chunk are we deleting ?
2008-09-12 02:55 <flips> ah, delete
2008-09-12 02:55 <flips> well, nice thing about truncate is, we don't have to wait for it
2008-09-12 02:55 <flips> we can just mark the inode as "truncated" and we're done
2008-09-12 02:56 <flips> we don't even have to update the inode
2008-09-12 02:56 <flips> just promise to in our log
2008-09-12 02:56 <bh> or do we need to lock during the read as well ?
2008-09-12 02:56 <flips> and take our sweek time, walking through the dtree, taking locks, freeing blocks
2008-09-12 02:56 <bh> on a block basis
2008-09-12 02:56 <flips> we need to lock on read, yes
2008-09-12 02:56 <flips> on a block basis
2008-09-12 02:56 <bh> which ? what does the lock hierarchy look like 
2008-09-12 02:56 <bh> ?
2008-09-12 02:57 <flips> just long enough to enter the block into the cache
2008-09-12 02:57 <bh> do we lock the itree ? dtree ? what ?
2008-09-12 02:57 <flips> we work our way down the levels of the two trees, taking locks and releasing them
2008-09-12 02:57 <flips> we only hold a lock long enough to know that we can see the next object in cache
2008-09-12 02:57 <bh> do we take reader locks along the way ?
2008-09-12 02:57 <flips> if we don't see it in cache, drop everything, read it in, start over fromthe top
2008-09-12 02:58 <flips> simple mined algorithm
2008-09-12 02:58 <flips> a starting point
2008-09-12 02:58 <bh> and hold them across the entire operation ?
2008-09-12 02:58 <bh> let's definte this
2008-09-12 02:58 <flips> only across the operation of finding the next level down in the cache
2008-09-12 02:58 <flips> soon as we find that, we lock it, release the parent
2008-09-12 02:58 <bh> be specific
2008-09-12 02:58 <flips> make sense?
2008-09-12 02:58 <flips> I thought that was specific
2008-09-12 02:59 <bh> hold the itree lock until we get the specific dtree ?
2008-09-12 02:59 <flips> there is no itree lock
2008-09-12 02:59 <bh> no, more specific :)
2008-09-12 02:59 <flips> ok, lock the root of the itree
2008-09-12 02:59 <bh> ok
2008-09-12 02:59 <flips> that is, look for it in cache
2008-09-12 02:59 <bh> ok
2008-09-12 02:59 <flips> if it's not there, issue a read
2008-09-12 02:59 <flips> block until it is
2008-09-12 02:59 <bh> to load the portion of the itree, right ?
2008-09-12 02:59 <flips> then block until we own the read lock
2008-09-12 02:59 <flips> have a read lock
2008-09-12 03:00 <flips> nope
2008-09-12 03:00 <bh> wait...
2008-09-12 03:00 <flips> to probe down where we want to go
2008-09-12 03:00 <bh> let me summarize
2008-09-12 03:00 <flips> yes, that stops everybody
2008-09-12 03:00 <flips> probing the itree
2008-09-12 03:00 <flips> well
2008-09-12 03:00 <flips> it stops writers
2008-09-12 03:00 <bh> yes
2008-09-12 03:00 <flips> because we have a read lock on the root
2008-09-12 03:00 <bh> right
2008-09-12 03:00 <flips> we aren't going to keep it long
2008-09-12 03:00 <flips> that would be unfriendly
2008-09-12 03:00 <bh> yes
2008-09-12 03:01 <flips> so what we do is, we find the next index block down inthe inode table index tree
2008-09-12 03:01 <flips> check its in cache
2008-09-12 03:01 <flips> if so, take a read lock on it
2008-09-12 03:01 <flips> if not, drop the root lock, issue a read, block on it, start again at the root
2008-09-12 03:01 <flips> obviously this may never terminate ;-)
2008-09-12 03:02 <flips> but we have other problems if it doesn't
2008-09-12 03:02 <bh> so we look up a inode; reader lock the itree; if it's not there issue a read to load that in, release all of the above locks until that block's wait queue wakes; read that block, get that dtree link
2008-09-12 03:02 <bh> while holding the itree reader lock
2008-09-12 03:02 <flips> we don't read lock the itree
2008-09-12 03:02 <bh> correct ?
2008-09-12 03:02 <flips> we read lock the root of the itree
2008-09-12 03:02 <flips> big difference
2008-09-12 03:02 <bh> ok
2008-09-12 03:02 <flips> so let's say the itree has seven levels of index
2008-09-12 03:02 <flips> big itree
2008-09-12 03:02 <bh> what's the difference ?
2008-09-12 03:02 <bh> ok
2008-09-12 03:03 <flips> we start by locking the root
2008-09-12 03:03 <flips> then we lock level one index, and drop the root lock
2008-09-12 03:03 <flips> then lock level 2 index block, and drop the level 1
2008-09-12 03:03 <flips> and so on
2008-09-12 03:03 <flips> down to level 7
2008-09-12 03:03 <flips> then we start the same process onthe dtree
2008-09-12 03:03 <flips> make sense?
2008-09-12 03:03 <bh> or propagate downwards, releasing a lock
2008-09-12 03:03 <bh> kind of scary
2008-09-12 03:03 <flips> ok, it's not that scary
2008-09-12 03:04 <flips> big reason: we wil normally keep hitting the same inode table block several times
2008-09-12 03:04 <flips> so we keep a "cursor"
2008-09-12 03:04 <bh> right, advance the cursor as needed
2008-09-12 03:04 <bh> what about rebalancing operations ? how does this effect it ?
2008-09-12 03:04 <flips> got to worry about how cursors interact with write locks on the itree
2008-09-12 03:04 <flips> but then
2008-09-12 03:04 <flips> that's why we're talking about it
2008-09-12 03:05 <bh> I don't know how to manipulate it other than with a big coarse grained lock as this time
2008-09-12 03:05 <flips> ok when you want to rebalance, delete, insert, split, whatever, you need a write lock
2008-09-12 03:05 <flips> on the parent and on the blocks being changed
2008-09-12 03:05 <flips> so you do the same thing
2008-09-12 03:05 <bh> I simply don't know enough about b-trees to know how to downward propagate the lock
2008-09-12 03:05 <bh> cursor
2008-09-12 03:05 <flips> me neither
2008-09-12 03:05 <flips> haven't done this before
2008-09-12 03:06 <flips> it's jsut brainwork though
2008-09-12 03:06 <flips> no magic
2008-09-12 03:06 <bh> ok, that's a big deal
2008-09-12 03:06 <bh> the expert on tree locking that I know of is peterz
2008-09-12 03:06 <flips> what kind of tree?
2008-09-12 03:06 <bh> he's done all sorts of shit
2008-09-12 03:06 <bh> radix tree and other things
2008-09-12 03:06 <flips> I'll ping him
2008-09-12 03:06 <bh> higly concurrent trees
2008-09-12 03:06 <flips> radix tree is pretty simple
2008-09-12 03:06 <bh> highly concurrent trees
2008-09-12 03:06 <flips> compared to a filesystem index
2008-09-12 03:07 <bh> he's the best person for the job that I know of
2008-09-12 03:07 <flips> yes, I'll point peterz at it
2008-09-12 03:07 <flips> you're not bad
2008-09-12 03:07 <flips> you're asking the right questions
2008-09-12 03:07 <bh> he might not have time, but I don't think that your current track of fine graining the system upfront is the best solution
2008-09-12 03:07 <flips> ah
2008-09-12 03:08 <flips> we have another knob we can tweak
2008-09-12 03:08 <bh> you should consider seriously per cpu-ing it if possible or faking it userspace
2008-09-12 03:08 <flips> there is also a refcount on each buffer
2008-09-12 03:08 <bh> we can get highly concurrent reads with RCU, that's a given
2008-09-12 03:08 <flips> per-cpuing it before figuring out how to do it with normal locks would not be wise
2008-09-12 03:08 <flips> 1) walk 2) run
2008-09-12 03:08 <bh> it's just a matter of how we can modify it to apply to your current atomic log at that time
2008-09-12 03:09 <flips> ok, see that recount comment
2008-09-12 03:09 <bh> maybe the use of an atomic counter would help to version the logs for both RCU tree nodes and the atomic disk log
2008-09-12 03:09 <flips> very important
2008-09-12 03:09 <flips> forget rcu
2008-09-12 03:09 <flips> rcu is braindamage
2008-09-12 03:10 <flips> when we want it rcu'd, we'll hand it to the rcu guys
2008-09-12 03:10 <bh> it can be but it's also a bad ass algorithm
2008-09-12 03:10 <flips> it's a real use of time
2008-09-12 03:10 <bh> depends on the kind of guarantees you need
2008-09-12 03:10 <flips> not consistent with getting a solid prototype up
2008-09-12 03:10 <bh> ok
2008-09-12 03:10 <bh> I guess basic thread safety is first
2008-09-12 03:10 <flips> anyway, the question, what happens when the itree geometry needs to change
2008-09-12 03:11 <flips> so we have all these readers walking down the tree, that's great
2008-09-12 03:11 <flips> and they release their locks as they go so somebody can come behind and maybe go down a different subtree
2008-09-12 03:11 <flips> very nice already
2008-09-12 03:11 <flips> but
2008-09-12 03:11 <flips> how do you change the tree geometry?
2008-09-12 03:11 <flips> well
2008-09-12 03:12 <flips> you can actuall do it when there are readers buzzing away inside subtrees that you're moving around
2008-09-12 03:12 <flips> that is cool
2008-09-12 03:12 <flips> you just need to write lock the parent and read lock the children, so you know that the tasks ahead of you have gotten off the parent
2008-09-12 03:13 <flips> then you can change the parent
2008-09-12 03:13 <flips> make sense?
2008-09-12 03:13 <flips> you can for example, split the parent
2008-09-12 03:13 <flips> and then you may have to change the parent's parent
2008-09-12 03:13 <flips> well
2008-09-12 03:13 <flips> fun
2008-09-12 03:14 <bh> 
ACTION reads

2008-09-12 03:14 <flips> you might have to check the path to find out how high up the splits will go, and get write locks on that whole chain
2008-09-12 03:14 <bh> I was just talking to peterz
2008-09-12 03:14 <bh> asked him the same questions we were talking about here
2008-09-12 03:14 <bh> he's not going to be around much since he's headed to plumber's
2008-09-12 03:14 <flips> the comparison of answers must be fascinating
2008-09-12 03:15 <bh> I think that itree node deletion needs to be tied to file handle semantics somehow
2008-09-12 03:15 <flips> tell him good luck with the plumbing, there is a lot of shit in those pipes
2008-09-12 03:15 <bh> na, forget that
2008-09-12 03:15 <flips> good
2008-09-12 03:15 <flips> it's way wrong ;)
2008-09-12 03:16 <bh> peterz used a lock in a linked list node to protect link modification
2008-09-12 03:16 <flips> so think about why your deleting an itree node
2008-09-12 03:16 <bh> yeah, you're completely removing it
2008-09-12 03:16 <flips> but why?
2008-09-12 03:16 <bh> it's not really what you want
2008-09-12 03:17 <bh> I'm agreening with you
2008-09-12 03:17 <flips> it's because you're coalescing the itree, and why are you doing that?
2008-09-12 03:17 <bh> agreeing
2008-09-12 03:17 <flips> I know
2008-09-12 03:17 <flips> I'm doing rhetoric
2008-09-12 03:17 <bh> ok
2008-09-12 03:17 <flips> so you're doing that because you've just delete masses of files and you want to tighten up the inode table tree a little
2008-09-12 03:17 <flips> actually, this is quite optional
2008-09-12 03:18 <flips> we don't really need to do that
2008-09-12 03:18 <flips> particularly if we tend to reuse the same inode numbers in the not too distant future
2008-09-12 03:18 <flips> it's only if we are determined to use completely different ones, for no good reason, that we need to fiddle the geometry of the itree on delete
2008-09-12 03:19 <flips> anway
2008-09-12 03:19 <flips> let's assume that we do want to be tidy and coalesce the itree frequently, even if we are not required to
2008-09-12 03:20 <flips> that means merging nodes in general
2008-09-12 03:20 <flips> just what I was talking about
2008-09-12 03:20 <flips> well
2008-09-12 03:20 <flips> no, I was thinking of splitting above
2008-09-12 03:20 <flips> merging is more forgiving as far as locking the access path goes
2008-09-12 03:20 <bh> tough problem
2008-09-12 03:21 <bh> I'd go with something simple first
2008-09-12 03:21 <flips> not really, you only need to write lock the parent and the two blocks being merged
2008-09-12 03:21 <bh> this is too much for a prototype
2008-09-12 03:21 <bh> but the first thing to do is to define specifically how a coarse grained set of locks would work on it
2008-09-12 03:21 <flips> it is traditional to start with a single global lock on each btree
2008-09-12 03:21 <bh> and then propagate downwards
2008-09-12 03:21 <flips> and find out how badly that sucks
2008-09-12 03:21 <bh> right
2008-09-12 03:21 <flips> we know in advance it sucks too much
2008-09-12 03:21 <flips> so why bother?
2008-09-12 03:21 <bh> we have lockstat so we can get a good idea of how sucky it is and it will be sucky
2008-09-12 03:21 <flips> maybe in next week's prototype
2008-09-12 03:21 <flips> then that's it
2008-09-12 03:22 <bh> ok
2008-09-12 03:22 <bh> the biggest focus at this time is to get your prototype working fully
2008-09-12 03:22 <flips> true
2008-09-12 03:22 <flips> with concurrency
2008-09-12 03:22 <flips> in user space
2008-09-12 03:22 <flips> had a slight change of philosophy there
2008-09-12 03:22 <bh> the best thing we can do is make provisions to do fine grained locking or per cpu-ification in the future easily
2008-09-12 03:22 <flips> when the fuse stuff landed
2008-09-12 03:22 <bh> not solve the entire problem upfront
2008-09-12 03:22 <flips> right
2008-09-12 03:22 <flips> well
2008-09-12 03:23 <flips> I don't think you mean by per-cpu what I mean
2008-09-12 03:23 <bh> but let's ask interesting questions and get help from folks like peterz
2008-09-12 03:23 <flips> per-cpu to me means replicating the relevant date per-cpu
2008-09-12 03:23 <flips> that's a big mess
2008-09-12 03:23 <flips> last resort
2008-09-12 03:23 <flips> all that
2008-09-12 03:24 <bh> no,
2008-09-12 03:24 <bh> it's about avoiding locking in the first place during an inode operation
2008-09-12 03:24 <flips> like how?
2008-09-12 03:24 <bh> as much locking as possible under that operation
2008-09-12 03:25 <bh> like making the entire read path as per cpu as possible
2008-09-12 03:25 <flips> vfs is going to take i_sem, can't tell it not to
2008-09-12 03:25 <flips> we have to stick to the part we own
2008-09-12 03:25 <bh> yeah
2008-09-12 03:25 <flips> and are responsible for
2008-09-12 03:25 <flips> which is our indexing structures
2008-09-12 03:25 <bh> so we're already taking an inode lock of some sort
2008-09-12 03:25 <flips> different inode
2008-09-12 03:26 <flips> there's the struct inode, which the vfs locks, and the image of the inode on an inode table block, which we lock
2008-09-12 03:41 <flips> sleeping time
2008-09-12 03:46 <bh> ok
2008-09-12 03:46 <bh> night
2008-09-12 03:46 <bh> later flips sleep well
2008-09-12 03:46 <bh> 
ACTION is going to be up still

2008-09-12 03:47 <flips> night
2008-09-12 08:44 -!- tim_dimm(~timothyhu@adsl-67-114-40-138.dsl.scrm01.pacbell.net) has joined #tux3
2008-09-12 09:10 -!- RazvanM(~RazvanM@dazzler.isi.jhu.edu) has joined #tux3
2008-09-12 09:26 -!- eli(~elicriffi@66.249.86.209) has joined #tux3
2008-09-12 10:24 -!- kd(kdpict@118.94.54.179) has joined #tux3
2008-09-12 11:15 -!- kushal(kdpict@118.94.54.179) has joined #tux3
2008-09-12 11:39 -!- pgquiles(~pgquiles@229.Red-83-49-101.dynamicIP.rima-tde.net) has joined #tux3
2008-09-12 13:05 <bh> flips: 
2008-09-12 13:05 <bh> 05:09 < peterz> bh: not too hard - I send you a paper on that iirc
2008-09-12 13:05 <bh> 05:09 < giel> not too hard implementation-wise, or time/space wise?
2008-09-12 13:05 <bh> 05:10 < giel> complexity theory!
2008-09-12 13:05 <bh> 05:10 < peterz> implementation wise :-)
2008-09-12 13:05 <bh> 05:10 < peterz> the btree space/time considerations don't change
2008-09-12 13:05 <bh> 05:11 < peterz> the thing that's hardest about the fine grain locking is the optimistic locking approach
2008-09-12 13:05 <bh> 05:11 < peterz> you'd have to work out where upwards traversal stops on your way down
2008-09-12 13:05 <flips> which channel?
2008-09-12 13:05 <flips> he's exactly right
2008-09-12 13:06 <flips> woke up thinking about precisely that
2008-09-12 13:06 <flips> this morning
2008-09-12 13:10 <bh> #offtopic2
2008-09-12 13:10 <bh> but he's traveling right now
2008-09-12 13:11 <bh> to KS and Plumbers
2008-09-12 13:11 <flips> it's not offtopic ;-)
2008-09-12 13:11 <flips> I'll invite peterz here
2008-09-12 13:11 <flips> better than #offtopic
2008-09-12 13:12 <flips> KS is going to be buzzing about tux3 ;-)
2008-09-12 13:12 <flips> lots of trash talking from the trash talkers
2008-09-12 13:13 <flips> KS has degenerated to mostly wanking
2008-09-12 13:13 <flips> very little tech gets done there any more
2008-09-12 13:13 <flips> just climbers getting fact time
2008-09-12 13:13 <bh> are they ?
2008-09-12 13:13 <bh> regarding buzz ?
2008-09-12 13:13 <flips> ?
2008-09-12 13:14 <flips> face time I mean
2008-09-12 13:14 <bh> oh are they a bunch of wankers now ? this is a publically logged channel keep in mind :)
2008-09-12 13:14 <flips> ah right
2008-09-12 13:14 <flips> well that's a public comment
2008-09-12 13:14 <bh> 
ACTION giggles

2008-09-12 13:14 <bh> not trying to make friends, eh ? :)
2008-09-12 13:14 <flips> never have gotten along well with wankers
2008-09-12 13:15 <flips> just me
2008-09-12 13:15 <bh> yeah, well, I can do with less political wanking and more changes into the kernel
2008-09-12 13:15 <flips> yep
2008-09-12 13:15 <flips> opensolaris helps focus on that
2008-09-12 13:15 <flips> not enough yet
2008-09-12 13:16 <bh> specificallly a couple of things I've been planning for years but never had the time to really do
2008-09-12 13:16 <flips> linux is losing "customers" to opensolaris
2008-09-12 13:16 <flips> it's a fact
2008-09-12 13:16 <bh> oh really ?
2008-09-12 13:16 <flips> not desktoppers, but datacenters
2008-09-12 13:16 <flips> backrooms
2008-09-12 13:16 <flips> the guys with money
2008-09-12 13:17 <bh> flips: have peterz resend you the paper, I don't know where it is for at the moment
2008-09-12 13:17 <flips> paper?
2008-09-12 13:18 <bh> the paper on the topic regarding trees and locking
2008-09-12 13:18 <flips> would be nice
2008-09-12 13:18 <bh> ask him for it
2008-09-12 13:18 <flips> sure
2008-09-12 13:19 <flips> we have to do some minor changes to btree.c I think
2008-09-12 13:19 <flips> because it currently climbs the path when it has to split
2008-09-12 13:19 <flips> it has to descend instead, and it has to drop locks before doing that
2008-09-12 13:20 <flips> so it may find that somebody else has changed the object is was looking at when it gets back down
2008-09-12 13:20 <flips> the object can't be deleted fortunately
2008-09-12 13:20 <flips> because its caller must hold a reference
2008-09-12 13:20 <bh> it'll have to be locked in-roder
2008-09-12 13:20 <bh> in-order
2008-09-12 13:20 <flips> on the cache image
2008-09-12 13:20 <bh> what ever that is
2008-09-12 13:20 <flips> so that is the rule: you have to hold a ref on the cached object before you can delete the disk object
2008-09-12 13:21 <flips> and the ref count of the cached object must be equal to one
2008-09-12 13:21 <flips> good example of unwritten lore about the kernel
2008-09-12 13:21 <flips> books don't tell you that
2008-09-12 13:21 <flips> but anybody who is allowed to touch core vfs knows that
2008-09-12 13:21 <flips> fs hackers have to know it do, and often don't
2008-09-12 13:21 <flips> know it too that is
2008-09-12 13:22 <flips> locking order for a btree is simple
2008-09-12 13:22 <flips> root-to-leaf
2008-09-12 13:22 <flips> left-to-right if that granularity matters, which it doesn't
2008-09-12 13:22 <flips> so just root-to-leaf
2008-09-12 13:23 <flips> but resize_btree goes leaf-to-root, doesn't work
2008-09-12 13:31 <flips> ok, time to stop cleaning up and write some refcounting code
2008-09-12 13:31 <flips> no comments back on my post last night
2008-09-12 13:31 <flips> I thought folks would chew on that
2008-09-12 13:32 <flips> it's really core to tux3 performance in general
2008-09-12 13:32 <flips> not just atom refcounting
2008-09-12 13:33 <data> hi. i just copied and pasted the irclogs from the tux3 university sessions. As I haven't seen them on the mailing list as of yet, should I post them?
2008-09-12 13:33 <flips> sure
2008-09-12 13:34 <flips> complete with all the swearing ;-)
2008-09-12 13:34 <flips> this is real life university
2008-09-12 13:34 <data> i'll make sure of it :)
2008-09-12 13:34 <flips> might replace some of the bad words with @%$@#
2008-09-12 13:34 <flips> or not
2008-09-12 13:34 <data> guess not :)
2008-09-12 13:34 <flips> whatever you think is right ;)
2008-09-12 13:34 <flips> nice nick
2008-09-12 13:35 <data> well, it was datapunk when I was like 15 or so
2008-09-12 13:35 <flips> also good
2008-09-12 13:35 <data> and as they tend to get shorte it now resembles a trekkie name
2008-09-12 13:35 <data> but thanks 
2008-09-12 13:35 <flips> you have piercings? or just virtual piercings?
2008-09-12 13:35 <data> just virtual
2008-09-12 13:36 <flips> :)
2008-09-12 13:36 <flips> some of my best friends in berlin had some interesting piercings
2008-09-12 13:36 <flips> but for example, harald avoids it
2008-09-12 13:36 <flips> works better in the boardroom
2008-09-12 13:36 <data> I was going to look at the reasons for it, but is the problem with deleting files known?
2008-09-12 13:37 <flips> frist I heard of it
2008-09-12 13:37 <flips> go ahead on it
2008-09-12 13:37 <data> well, i don't particularly like them
2008-09-12 13:37 <flips> I wasn't very careful when I put that in
2008-09-12 13:37 <data> ok, will do, after a little algebra session
2008-09-12 13:37 <data> see you later
2008-09-12 13:37 <flips> wo wohnst du?
2008-09-12 13:37 <data> karlsruhe
2008-09-12 13:37 <data> if you know it
2008-09-12 13:37 <flips> ah cool
2008-09-12 13:37 <flips> near SAS
2008-09-12 13:37 <flips> sure, been there a few times
2008-09-12 13:38 <flips> quite
2008-09-12 13:38 <flips> quiet
2008-09-12 13:38 <flips> just like the name
2008-09-12 13:38 <flips> lots of geeks in the area
2008-09-12 13:38 <data> yep, they are
2008-09-12 13:38 <data> CS is pretty strong
2008-09-12 13:38 <flips> suse not far away
2008-09-12 13:38 <flips> ibm
2008-09-12 13:38 <flips> not sas
2008-09-12 13:38 <flips> um
2008-09-12 13:38 <flips> um
2008-09-12 13:38 <data> sap?
2008-09-12 13:38 <flips> right
2008-09-12 13:38 <flips> where I've been too
2008-09-12 13:39 <flips> there's a great guy there
2008-09-12 13:39 <data> gotten around a lot?
2008-09-12 13:39 <flips> drei jahre in Deutscheland
2008-09-12 13:39 <flips> um, 6 jahre
2008-09-12 13:40 <flips> that would be 6 Jahre
2008-09-12 13:40 <flips> getting rusty
2008-09-12 13:40 <data> for work i guess?
2008-09-12 13:40 <flips> and fun
2008-09-12 13:40 <data> well, i've only been to the usa for 11 months
2008-09-12 13:40 <data> and that was for school... and fun
2008-09-12 13:41 <flips> that's about enough to be honest
2008-09-12 13:41 <flips> berlin is a lot more fun
2008-09-12 13:41 <flips> and less tense
2008-09-12 13:41 <flips> contrary to popular belief
2008-09-12 13:41 <data> only been there a few times, mostly during the ccc congresses
2008-09-12 13:41 <data> but it certainly is fun
2008-09-12 13:41 <flips> geek hotbed
2008-09-12 13:42 <data> ok gotta go, will be back in an hour or so
2008-09-12 13:42 <flips> hottest hotbed in europe imho
2008-09-12 13:42 <flips> bis spater dann
2008-09-12 13:42 <data> und zu weit weg :)
2008-09-12 13:42 <data> bis dann
2008-09-12 13:42 <flips> 
ACTION is getting really rusty ;)

2008-09-12 13:44 <flips> I just had a thought
2008-09-12 13:44 <flips> we should schedule an official tux3 cabal meeting for Oct 31
2008-09-12 13:44 <flips> on irc, plus a real location in LA
2008-09-12 13:47 <bh> might be a good time
2008-09-12 14:12 -!- tim_dimm(~timothyhu@adsl-67-114-40-138.dsl.scrm01.pacbell.net) has joined #tux3
2008-09-12 14:12 <flips> hi tim_dimm
2008-09-12 14:12 <flips> coming up for air?
2008-09-12 14:12 <tim_dimm> hi flips
2008-09-12 14:12 <tim_dimm> trying to
2008-09-12 14:12 <flips> manage a quick skate today?
2008-09-12 14:13 <tim_dimm> still in sacramento
2008-09-12 14:13 <flips> oh right
2008-09-12 14:13 <tim_dimm> Pi and Persey are still in the ICU
2008-09-12 14:13 <flips> and it would be inadvisable anyway
2008-09-12 14:13 <tim_dimm> how's that?
2008-09-12 14:13 <flips> you got a week with no weeks under you to look forward to
2008-09-12 14:13 <tim_dimm> heh
2008-09-12 14:13 <flips> can't justify skating, unless it is to the nursery
2008-09-12 14:14 <flips> with no wheels under you I meant
2008-09-12 14:14 <tim_dimm> I can justify it if the heart rate is up enough
2008-09-12 14:14 <flips> getting bad typoitis here
2008-09-12 14:14 <flips> full word typos now
2008-09-12 14:14 <tim_dimm> happens when the volume of code goes stratospheric
2008-09-12 14:14 <flips> just read through the first of two tux3 U sessions
2008-09-12 14:14 <flips> it hung together pretty well
2008-09-12 14:14 <tim_dimm> going to now
2008-09-12 14:15 <flips> not too much pure bs
2008-09-12 14:15 <flips> some content
2008-09-12 14:15 <tim_dimm> what time of day did you start?
2008-09-12 14:15 <flips> today?
2008-09-12 14:15 <flips> or last night?
2008-09-12 14:15 <flips> 8 pm tue and thur
2008-09-12 14:15 <tim_dimm> no, for the U
2008-09-12 14:15 <tim_dimm> k
2008-09-12 14:15 <flips> will be regular
2008-09-12 14:15 <flips> as far as I can manage
2008-09-12 14:15 <tim_dimm> sounds like a great tool for building community
2008-09-12 14:15 <flips> we had one inner linux guru here thursday
2008-09-12 14:16 <flips> eric biederman
2008-09-12 14:16 <flips> linux cluster guy
2008-09-12 14:16 <flips> the linux cluster guy
2008-09-12 14:16 <flips> and of course natalie is an inner linux gal
2008-09-12 14:16 <tim_dimm> googling...
2008-09-12 14:16 <flips> you'll get a few hits ;-)
2008-09-12 14:17 <tim_dimm> you get that email this am about SE Linux?
2008-09-12 14:17 <flips> 223K to be exact
2008-09-12 14:18 <flips> no
2008-09-12 14:18 <flips> let me check
2008-09-12 14:18 <flips> oh
2008-09-12 14:18 <flips> yes
2008-09-12 14:19 <tim_dimm> right direction?
2008-09-12 14:19 <flips> knew about apparmor, suse's better answer to selinux
2008-09-12 14:19 <flips> uses the same kernel hooks
2008-09-12 14:19 <flips> yes
2008-09-12 14:19 <flips> I'm not sure how much apparmor is being worked on right now
2008-09-12 14:19 <flips> it's another one of those good projects that gets beaten up by something sloppier but more devs
2008-09-12 14:20 <tim_dimm> apparently, Novell canned all the engineers working on it in '07
2008-09-12 14:20 <tim_dimm> bh...Got any intel on apparmor?
2008-09-12 14:20 <flips> ?
2008-09-12 14:20 <flips> ah
2008-09-12 14:20 <flips> 
ACTION pokes bh

2008-09-12 14:21 <tim_dimm> heh
2008-09-12 14:21 <tim_dimm> be nice to know what happened there
2008-09-12 14:21 <flips> see who's maintaining it even
2008-09-12 14:21 <flips> somebody always maintains os projects
2008-09-12 14:21 <flips> they never die... except for evms
2008-09-12 14:21 <flips> RIP
2008-09-12 14:21 <flips> well
2008-09-12 14:22 <flips> lvm3 will rise ;-)
2008-09-12 14:22 <flips> we're about a month away from serious lvm3 development
2008-09-12 14:22 <tim_dimm> http://www.novell.com/linux/security/apparmor/selinux_comparison.html
2008-09-12 14:22 <flips> kickoff
2008-09-12 14:22 <flips> tim_dimm, there's a proposal to have a public tux3 cabal meeting on Oct 31
2008-09-12 14:22 <flips> physically located at a certain garage I'm thinking of
2008-09-12 14:23 <flips> and on the web/net
2008-09-12 14:23 <flips> what think you?
2008-09-12 14:23 <tim_dimm> I'm there barring spit-up, diaper changes and burping sessions
2008-09-12 14:23 <flips> barring?
2008-09-12 14:23 <flips> should be irrespective of
2008-09-12 14:23 <tim_dimm> uh, poor choice of words
2008-09-12 14:23 <flips> anything except burping
2008-09-12 14:23 <tim_dimm> farting?
2008-09-12 14:23 <flips> not good enough
2008-09-12 14:23 <tim_dimm> too early to teeth
2008-09-12 14:24 <tim_dimm> how do you spell that- teeeethhh
2008-09-12 14:24 <tim_dimm> you know
2008-09-12 14:24 <flips> dana had one the first week
2008-09-12 14:24 <flips> was hell for anna
2008-09-12 14:24 <tim_dimm> i bet
2008-09-12 14:24 <flips> she quickly learned how to punish mommy with it
2008-09-12 14:25 <flips> so began a somewaht tense relationship ;)
2008-09-12 14:25 <tim_dimm> still?
2008-09-12 14:25 <tim_dimm> ;-)
2008-09-12 14:25 <flips> of course
2008-09-12 14:25 <flips> but detent has set in, mutual respect, mommy love, all that
2008-09-12 14:25 <flips> this apparently lasts till about 9 YO
2008-09-12 14:26 <flips> with luck
2008-09-12 14:26 <tim_dimm> guess they grow out of it
2008-09-12 14:26 <flips> tween is the new teen
2008-09-12 14:26 <flips> anyway
2008-09-12 14:26 <flips> we better stop talking like that
2008-09-12 14:26 <flips> or all the devs willrun away screaming
2008-09-12 14:27 <tim_dimm> tux3 and child-rearing 
2008-09-12 14:27 <tim_dimm> and skating
2008-09-12 14:27 <flips> you will have lots of time to learn C while you're burping
2008-09-12 14:27 <flips> and rocking
2008-09-12 14:27 <flips> right, on that note I think I'll go skate now
2008-09-12 14:27 <flips> oh, big news
2008-09-12 14:27 <tim_dimm> k
2008-09-12 14:28 <tim_dimm> all ears
2008-09-12 14:28 <tim_dimm> (eyes)
2008-09-12 14:28 <flips> skateboarders clapped for my move yesterday
2008-09-12 14:28 <tim_dimm> nice- what was it?
2008-09-12 14:28 <flips> pronounced: "ok rollerbladers are allowed now"
2008-09-12 14:28 <flips> nothing much
2008-09-12 14:28 <tim_dimm> grind?
2008-09-12 14:28 <flips> skated up on the little vert wall on one skate, tapped the top with the other, skated down on one skate
2008-09-12 14:29 <tim_dimm> nice
2008-09-12 14:29 <flips> been grinding and getting nodes
2008-09-12 14:29 <flips> also skating down the top of the grinding wall
2008-09-12 14:29 <flips> very skinny
2008-09-12 14:29 <flips> tough to stay on
2008-09-12 14:29 <flips> it has an S curve at the end
2008-09-12 14:29 <tim_dimm> careful, grinds lead to crashes which leads to wrist injuries
2008-09-12 14:29 <flips> not much, but enough to drop you off
2008-09-12 14:29 <flips> my grinds aren't really grinds
2008-09-12 14:30 <flips> just slding down the rail onthe side of my skate
2008-09-12 14:30 <flips> one foot
2008-09-12 14:30 <flips> no danger
2008-09-12 14:30 <flips> I need to get protection before doing anything more
2008-09-12 14:30 <flips> makes lots of noise
2008-09-12 14:30 <flips> attracts attention ;)
2008-09-12 14:31 <tim_dimm> found the head of the U logs
2008-09-12 14:31 <tim_dimm> reading now
2008-09-12 14:31 <flips> have fun
2008-09-12 14:31 <tim_dimm> loads
2008-09-12 15:08 <data> i just noticed: someone (?) said that dentries were 132 bytes. on my system it says 200. Normal deviations?
2008-09-12 15:09 <data> or just a different kernel version?
2008-09-12 15:10 <tim_dimm> I'm reading the logs right now, see 8 references to dentries. none mention how many bytes
2008-09-12 15:11 <data> second.05:29 < RazvanM> dentry            253015 253576    132   29    1 : tunables  120   60    8 : slabdata   8744   8744      0
2008-09-12 15:12 <tim_dimm> my bad- I searched for dentries
2008-09-12 15:12 <tim_dimm> not dentry
2008-09-12 15:13 <data> well, not really important. just something i was wondering about
2008-09-12 15:14 <flips> data, 64 bit kernel?
2008-09-12 15:15 <flips> grossly big aren't they
2008-09-12 15:15 <flips> filename "foo" turns into a 200 byte dentry, and that's far from all the cache gobbling for that little guy
2008-09-12 15:15 <data> yes, it is. right on both accounts.
2008-09-12 15:16 <flips> that's what makes sysfs such an idiotic idea
2008-09-12 15:16 <flips> take tiny little ascii strings which are already bloated way beyond the binary rep, and blow them up into gigantic, slow, awkward things
2008-09-12 15:17 <flips> then implement it badly on top of that
2008-09-12 15:17 <flips> and have a crappy internal and external interface
2008-09-12 15:17 <flips> bugs
2008-09-12 15:17 <flips> unstable api
2008-09-12 15:17 <flips> and you have the piece of shit we see today
2008-09-12 15:17 <flips> just thought I'd share that ;-)
2008-09-12 15:50 <flips> sk8 oclock
2008-09-12 15:54 <bh> 
ACTION is back

2008-09-12 15:54 <bh> I know nothing about apparmor
2008-09-12 15:57 <tim_dimm> kernel klink I presume ;-)
2008-09-12 15:57 <tim_dimm> (hogan's hero's)
2008-09-12 15:58 <tim_dimm> diaper-30
2008-09-12 15:58 <tim_dimm> l8tr
2008-09-12 18:17 <flips> nuther cuppa
2008-09-12 18:17 <flips> should be good enough to get refcounting implemented
2008-09-12 18:32 <flips> ok I see october 31st is a friday
2008-09-12 18:32 <flips> that means that the tux3 cabal meeting has to be a party
2008-09-12 18:32 <flips> might have to scale this up
2008-09-12 19:04 <flips> #define REFCOUNT_TABLE_BLOCK (1ULL << 28)
2008-09-12 19:04 <flips> #define REFCOUNT_HIGH_BLOCK (REFCOUNT_TABLE_BLOCK + (1ULL << 21))
2008-09-12 19:04 <flips> #define UNATOM_TABLE_BLOCK (REFCOUNT_TABLE_BLOCK + (1ULL << 23))
2008-09-12 19:10 -!- Aks(~ankitsriv@123.237.71.198) has joined #tux3
2008-09-12 19:16 <Aks> Hi all, I am new to this project and want to know abt versioned pointers.
2008-09-12 19:16 <flips> welcome
2008-09-12 19:16 <flips> read the post yet?
2008-09-12 19:17 <flips> http://lwn.net/Articles/288896/
2008-09-12 19:17 <Aks> thanks for the link
2008-09-12 19:18 <flips> enjoy
2008-09-12 19:22 <flips> 	atom = dir->sb->atomgen++; /* use refcount for allocation */
2008-09-12 19:22 <flips> 	if (!ext2_create_entry(dir, name, len, atom, 0)) {
2008-09-12 19:22 <flips> 		unsigned block = ATOM_REFCOUNT_BLOCK + (atom >> (dir->sb->blockbits - 1));
2008-09-12 19:22 <flips> 		struct buffer *buffer = bread(dir->map, block);
2008-09-12 19:22 <flips> 		*(u16 *)buffer->data += 1;
2008-09-12 19:22 <flips> 		brelse(buffer);
2008-09-12 19:22 <flips> 		return atom;
2008-09-12 19:22 <flips> 	}
2008-09-12 19:23 <flips> got to put in the carry bit handling
2008-09-12 19:29 <flips> mistake in that code
2008-09-12 19:30 <flips> 
ACTION challenges tux3 readers to find it

2008-09-12 19:30 <flips> it's in the block calc
2008-09-12 19:32 <flips> 
ACTION steps out for a bit

2008-09-12 19:32 <flips> oh, and I have to do endian conversion
2008-09-12 19:32 <flips> almost forgot
2008-09-12 20:20 <MaZe> atom >> (dir->sb->blockbits - 1) - that looks weird, although I'm not actually sure what exactly blockbits is, either way the -1 smells wrong
2008-09-12 20:21 <MaZe> ah, never mind 
2008-09-12 20:21 <MaZe> *(u16 *)buffer->data += 1 <- this is wrong since this is first u16 in block
2008-09-12 20:23 <MaZe> lacks [atom & ((1 << (dir->sb->blockbits)) - 1] instead of the prefix '*'
2008-09-12 20:23 <MaZe> ie. it should be ((u16 *)buffer->data)[atom & ((1 << (dir->sb->blockbits)) - 1]++;
2008-09-12 20:24 <MaZe> ((u16 *)buffer->data)[atom & ((1 << dir->sb->blockbits) - 1)]++;
2008-09-12 20:24 <MaZe> parentheses got mixed up
2008-09-12 20:25 <MaZe> from which I guess blockbits on a 4K filesystem is lg2(4Ki) = 12
2008-09-12 20:26 <MaZe> at which point the first comment about it smelling funny is irrelevant (I thought blockbits was the number of bits in a block, ie. 4Ki * 8 for a 4KiB block)
2008-09-12 20:26 <MaZe> so you're using bread/brelse, not bios?
2008-09-12 20:26 <MaZe> does brelse bwrite?
2008-09-12 20:27 <MaZe> I guess it must work like some sort of in kernel mmap
2008-09-12 20:27 <MaZe> hence the no nead for explicit write back
2008-09-12 20:27 <MaZe> at which point I guess we rely on cpu page dirty bits to actually know whether we need to write back
2008-09-12 20:31 <MaZe> unless bread/brelse, are actually operations on blocks within a file, which is suggested by the dir->map first parameter
2008-09-12 20:31 <MaZe> hmm, isn't it clear I have no bloody idea what I'm talking about yet?
2008-09-12 20:31 <MaZe> and I'm talking into a blackhole...
2008-09-12 20:31 <MaZe> start talking to yourself... then you know you're going crazy.
2008-09-12 20:41 -!- Kirantpatil(~kiran@122.167.202.116) has joined #tux3
2008-09-12 21:08 <flips> maze, it's all messed up, actually
2008-09-12 21:08 <flips> rev on the way
2008-09-12 21:08 <flips> you are right about the lacks
2008-09-12 21:08 <flips> maze, I should just have asked you to write it ;-)
2008-09-12 21:08 <flips> 
ACTION will be back

2008-09-12 21:09 <MaZe> which lacks
2008-09-12 21:09 <MaZe> don't leave ;-) I'm here
2008-09-12 21:09 <MaZe> what am I right about? I was guessing above?
2008-09-12 21:10 <MaZe> ofcourse with u16* in their it's all only host-endian compatible
2008-09-12 21:10 <MaZe> s/their/there
2008-09-12 21:11 <MaZe> I think stick to 1 byte counters - deal away with endianness in all but 0.5% of the cases
2008-09-12 21:12 <MaZe> eh, I'm not sure this entire effort is worth it
2008-09-12 21:12 <MaZe> I think you just need to support a half-dozen hard coded atoms, a small list of atoms for selinux, and store the rest as strings
2008-09-12 21:12 <MaZe> ultimately just gzipping the xattr block may be the easiest
2008-09-12 21:13 <MaZe> for everything besides selinux/acl
2008-09-12 21:13 <MaZe> [still parsing through the binary acl encoding, to see if it can be faked]
2008-09-12 21:20 -!- nataliep(~nataliep@207.47.98.129.static.nextweb.net) has joined #tux3
2008-09-12 21:33 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #tux3
2008-09-12 21:35 <MaZe> although linux currently seems to use something more like 32 [4 byte header] + (16 [tag/type] + 16[...rwx] + 32[default=-1]) * 4 + (16 [tag/type] + 16[...rwx] + 32[uid/gid]) * [# of exceptions] bits
2008-09-12 21:35 <MaZe> either way, while these are normally small - or not even present - they can grow arbitrally large
2008-09-12 21:37 <MaZe> hmm
2008-09-12 21:37 <MaZe> interesting questions
2008-09-12 21:37 <MaZe> do filesystems in linux implement selinux and acls, or do they just implement xattr - would think just xattr, but... there are ext2/3/4... etc acl.h
2008-09-12 21:38 <MaZe> oh, directories get doubled entries, one being default
2008-09-12 21:38 <MaZe> the other being actual
2008-09-12 21:39 <MaZe> the 4 byte header is the version (lendian 2)
2008-09-12 22:00 <flips> back
2008-09-12 22:00 <flips> maze, it's not much effort
2008-09-12 22:00 <MaZe> .
2008-09-12 22:00 <flips> and it exactly emulates the ascii xattr behaviour, with superior compression
2008-09-12 22:01 <flips> now I just need to write it right ;)
2008-09-12 22:01 <MaZe> yes, but there are some issues there
2008-09-12 22:01 <flips> there are?
2008-09-12 22:01 <MaZe> for example: selinux is a quad of four values
2008-09-12 22:01 <flips> I thought I was just about done
2008-09-12 22:01 <MaZe> if you have a lot of valid states for each of them
2008-09-12 22:01 <MaZe> then the total number of states for the entire quad can blossom
2008-09-12 22:01 <flips> those are attr bodies
2008-09-12 22:02 <flips> we're working on attr names at the moment
2008-09-12 22:02 <MaZe> ah, and see... this is the tricky part
2008-09-12 22:02 <MaZe> you kind of have to look at some of them at the same time
2008-09-12 22:02 <flips> some of... selinux acls?
2008-09-12 22:03 <MaZe> so basically, once you strip out the selinux and extended acl xattrs (that's 3 different xattr strings), all that's left is barely used by anyone
2008-09-12 22:03 <flips> ok I see what you're saying
2008-09-12 22:03 <flips> sorry, didn't read carefully
2008-09-12 22:03 <flips> you're running out way ahead of me
2008-09-12 22:03 <flips> as usual
2008-09-12 22:03 <MaZe> it may not be worth optimizing that...
2008-09-12 22:03 <flips> well it can be optimized at the selinux level
2008-09-12 22:04 <MaZe> exactly.
2008-09-12 22:04 <flips> maybe not quite as efficiently
2008-09-12 22:04 <flips> maybe more
2008-09-12 22:04 <MaZe> so selinux basically needs to be optimized
2008-09-12 22:04 <flips> let's let them tell us
2008-09-12 22:04 <MaZe> acls need to be optimized
2008-09-12 22:04 <flips> probably
2008-09-12 22:04 <MaZe> and then all the rest needs to be (maybe?) optimized if we feel like it
2008-09-12 22:04 <flips> so I suggest that once xattrs are working properly, we invite the selinux folks to come over and do an audit
2008-09-12 22:04 <MaZe> and what needs to be optimized is not the headers (ie. the security.something= part)
2008-09-12 22:04 <MaZe> but the bodies
2008-09-12 22:05 <MaZe> ie. the part after the =
2008-09-12 22:05 <flips> both
2008-09-12 22:05 <flips> imho
2008-09-12 22:05 <MaZe> the part before the = sign is trivial, there are about 9 values to compress as atoms, leave the rest as strings
2008-09-12 22:05 <flips> the heads are much less variable, therefore so much easier to optimize
2008-09-12 22:05 <MaZe> agreed.
2008-09-12 22:05 <MaZe> the problem, though, isn't so much what to optimize
2008-09-12 22:06 <flips> so... bodies
2008-09-12 22:06 <flips> not that hard
2008-09-12 22:06 <MaZe> but how to store this, and where...
2008-09-12 22:06 <flips> but maybe not appropriate at this level
2008-09-12 22:06 <flips> we'll see
2008-09-12 22:06 <MaZe> and which parts to store on disk where
2008-09-12 22:06 <MaZe> it's kind of important to plan this out correctly to begin with, because this is ondisk format, not in memory
2008-09-12 22:06 <flips> lyou know, we'd probably get more mileage out of giving the selinux guys a way to run their own dictionary
2008-09-12 22:06 <flips> just like our atom dictionary
2008-09-12 22:06 <MaZe> exactly...
2008-09-12 22:06 <flips> and have an api for it
2008-09-12 22:06 <flips> kay
2008-09-12 22:06 <flips> we'll propose it
2008-09-12 22:06 <MaZe> I'm convinced now we need 4 dicts for selinux (one per quad)
2008-09-12 22:07 <flips> but first let them have a look at the basics
2008-09-12 22:07 <MaZe> and a dictionary for acls
2008-09-12 22:07 <MaZe> and a dictionary for 'other' xattrs headers
2008-09-12 22:07 <flips> it's my understanding they're usually disappointed with performance etc of just basic xattrs
2008-09-12 22:07 <MaZe> so basically what we need is some extensible dict interface
2008-09-12 22:07 <flips> trying not to let tux3 fall into that
2008-09-12 22:07 <MaZe> which falls into the log nicely
2008-09-12 22:07 <flips> right
2008-09-12 22:07 <flips> and it's app specific
2008-09-12 22:07 <flips> we need to export an api, not for dicts
2008-09-12 22:07 <flips> not four dicts
2008-09-12 22:08 <MaZe> the api would be the normal add/remove/list xattr api that everybody uses
2008-09-12 22:08 <MaZe> what's important is how we store it internally in the fs
2008-09-12 22:08 <flips> plus a wahy of dividing it into four
2008-09-12 22:08 <MaZe> that's easy - split on :
2008-09-12 22:08 <flips> bleah
2008-09-12 22:08 <flips> no parsing
2008-09-12 22:08 <flips> in the fs
2008-09-12 22:09 <flips> mechanism, not policy
2008-09-12 22:09 <MaZe> yeah, well, that's gonna have to happen, unless you don't want to split the quads
2008-09-12 22:09 <MaZe> which could potentiall explode the dics
2008-09-12 22:09 <flips> no parsing ;-)
2008-09-12 22:09 <flips> notice, tux3 has no parsing
2008-09-12 22:09 <flips> if we want we can use zlib
2008-09-12 22:09 <flips> or something more global
2008-09-12 22:09 <MaZe> you're missing the point here ;-)
2008-09-12 22:09 <MaZe> zlib is nice and all that
2008-09-12 22:10 <MaZe> but selinux xattrs are used on every frickin file access
2008-09-12 22:10 <flips> still
2008-09-12 22:10 <MaZe> they have to be blazing fast
2008-09-12 22:10 <flips> no parsing ;-)
2008-09-12 22:10 <flips> we need a better solution
2008-09-12 22:10 <MaZe> you can make the parsing in such a way that it'll still work even if it doesn't parse
2008-09-12 22:10 <flips> preferably one that performs even better than stupid ascii colon separated strings
2008-09-12 22:10 <MaZe> ah, but the ascii colon seperated strings are the api
2008-09-12 22:11 <MaZe> you have to do it that way
2008-09-12 22:11 <flips> sucky api
2008-09-12 22:11 <MaZe> unless we rip through all of the selinux code in the kernel
2008-09-12 22:11 <flips> anyway, linux does not have an acl api
2008-09-12 22:11 <flips> selinux does
2008-09-12 22:11 <flips> different
2008-09-12 22:11 <MaZe> the vfs layer provides an xattr api
2008-09-12 22:11 <MaZe> that's what we have to implement
2008-09-12 22:11 <flips> right, only that
2008-09-12 22:11 <flips> well
2008-09-12 22:11 <flips> we don't have to parse
2008-09-12 22:11 <MaZe> however internally we have to make it deal with the common cases quickly
2008-09-12 22:12 <flips> we can compress on byte pair if we like
2008-09-12 22:12 <MaZe> byte pair of what? you're getting strings in the api?
2008-09-12 22:12 <flips> byte pairs is a typical compression method
2008-09-12 22:12 <flips> 16 bit values work better with a dict than 8 or 48
2008-09-12 22:12 <flips> for example
2008-09-12 22:12 <MaZe> and the common cases are going to be reading (and to a lesser extent writing) selinux xattrs and less often (but still very often) extended acls
2008-09-12 22:13 <flips> verging on premature optimization here
2008-09-12 22:13 <MaZe> no no, don't think lzw compression - that doesn't buy us anything here
2008-09-12 22:13 <flips> the selinux guys will cream all over if xattrs just work fine
2008-09-12 22:13 <MaZe> what needs to be done is we need to explicitly remove the selinux/acl from the xattr code and not treat them in the fs as xattrs at all
2008-09-12 22:13 <MaZe> treat them like you treat inode permissions
2008-09-12 22:14 <MaZe> put them directly in the inode
2008-09-12 22:14 <flips> have you measured the actual disbribution of unique quads?
2008-09-12 22:14 <flips> I thought you did that
2008-09-12 22:14 <MaZe> yes
2008-09-12 22:14 <flips> and it came out very tight
2008-09-12 22:14 <MaZe> exactly
2008-09-12 22:14 <flips> tightly clustered too
2008-09-12 22:14 <flips> so what's the problem
2008-09-12 22:14 <MaZe> but that's not something we can guarantee on a prod system
2008-09-12 22:14 <flips> just atomize the common ones
2008-09-12 22:14 <flips> store the weirdos literally
2008-09-12 22:14 <MaZe> agreed.
2008-09-12 22:14 <flips> ok
2008-09-12 22:14 <flips> so let's do it
2008-09-12 22:15 <MaZe> hmm, how to put this
2008-09-12 22:15 <MaZe> you don't want the xattr_get(selinux_xattr)
2008-09-12 22:15 <MaZe> to have to parse the entire xattr block for the inode
2008-09-12 22:16 <flips> it doesn't
2008-09-12 22:16 <flips> it only looks in the xcache
2008-09-12 22:16 <MaZe> but xattrs of other types, can be pretty much unique per file...
2008-09-12 22:16 <MaZe> (above md5/sha1 hash case)
2008-09-12 22:17 <flips> right
2008-09-12 22:17 <flips> so I guess we're going to check a hash of the xattr
2008-09-12 22:17 <flips> htree style
2008-09-12 22:17 <MaZe> so having the two very differently performing/characteristic concepts in one place  will most likely break performance
2008-09-12 22:18 <flips> it's not very hard to look for likely atomize candidates I think
2008-09-12 22:18 <MaZe> depends on what the symbols of the alphabet
2008-09-12 22:18 <MaZe> and how deeply you're parsing
2008-09-12 22:18 <flips> I say, just put everything in the dict
2008-09-12 22:18 <flips> why not?
2008-09-12 22:18 <flips> has to be stored somewhere
2008-09-12 22:19 <MaZe> security.selinux="unconfined_u:object_r:default_t:s0\000"
2008-09-12 22:19 <MaZe> system.posix_acl_access="0sAgAAAAEABwD/////AgAEAGQAAAAEAAUA/////xAABQD/////IAAFAP////8="
2008-09-12 22:19 <MaZe> user.hash="sdfsdfjhsdjfhsdjkfhdjskahfjkdsahkj"
2008-09-12 22:19 <flips> the dict is as good a place as any
2008-09-12 22:19 <MaZe> how would you atomize the above?
2008-09-12 22:19 <flips> ext2_find_entry
2008-09-12 22:19 <flips> later, htree_find_entry
2008-09-12 22:20 <flips> sucky compression in the example
2008-09-12 22:21 <MaZe> so how will the dict deal, with a few dozen entries with milions of occurences, a few hundred with tens of thousands, and a few million entries with one (to a couple) occurence(s) each
2008-09-12 22:21 <MaZe> that's a real world scenario straight of my laptop 20G drive
2008-09-12 22:21 <flips> millions of occurences, what's the problem?
2008-09-12 22:21 <flips> few million, easy
2008-09-12 22:21 <flips> that's what htree does
2008-09-12 22:21 <flips> handles millions of entries
2008-09-12 22:21 <flips> really fast
2008-09-12 22:22 <MaZe> uhm, I think my problem is I'm not convinced it's fast enough, when it could be O(1)
2008-09-12 22:22 <flips> o(1) is always good
2008-09-12 22:22 <MaZe> the millions of unique entries are blossoming the tree
2008-09-12 22:22 <flips> but damm fast is damm fast
2008-09-12 22:23 <flips> it's a btree
2008-09-12 22:23 <MaZe> slowing down accesses for the millions of entries
2008-09-12 22:23 <flips> it likes to blossom
2008-09-12 22:23 <flips> it says "go ahead, make my day"
2008-09-12 22:23 <MaZe> right, but are common entries stored nearer the root?
2008-09-12 22:23 <flips> never
2008-09-12 22:23 <MaZe> no - because it's a btree
2008-09-12 22:23 <flips> right
2008-09-12 22:23 <MaZe> so you've got o(depth) lookups
2008-09-12 22:23 <flips> very flat
2008-09-12 22:23 <flips> usually only two levels
2008-09-12 22:23 <MaZe> precisely what you want to avoid
2008-09-12 22:23 <flips> for a few million entires
2008-09-12 22:23 <flips> depth is smaller than you think
2008-09-12 22:24 <flips> much smaller
2008-09-12 22:24 <MaZe> but you're thinking of access speed from a disk io performance
2008-09-12 22:24 <MaZe> outlook
2008-09-12 22:24 <flips> nope
2008-09-12 22:24 <MaZe> we need to be fast in ram
2008-09-12 22:24 <flips> cpu speed
2008-09-12 22:24 <flips> that's what it works at
2008-09-12 22:24 <flips> dirops are cpu bound
2008-09-12 22:24 <flips> not disk bound
2008-09-12 22:25 <MaZe> ok, I'm firmly of the opinion we need 2 different dicts/htrees at the minimum
2008-09-12 22:25 <flips> I agree: 1) heads 2) bodies
2008-09-12 22:25 <flips> but I know you want to parse and segment
2008-09-12 22:25 <MaZe> one small one for the stuff which is known to exist all over the place (selinux/acl bodies)
2008-09-12 22:25 <flips> I don't think we should, the selinux guys should
2008-09-12 22:25 <flips> but
2008-09-12 22:25 <MaZe> the other for non-standard bodies
2008-09-12 22:25 <flips> I;'ll keep an open mind
2008-09-12 22:25 <MaZe> we would need to change the kernel vfs interface of selinux - I don't see that happening
2008-09-12 22:26 <MaZe> and then we'd need to keep around the old one for other fs'es anyway
2008-09-12 22:26 <flips> well xattrs already have namespaces
2008-09-12 22:26 <flips> part of the abpi
2008-09-12 22:26 <flips> api
2008-09-12 22:26 <flips> braindamaged part
2008-09-12 22:26 <MaZe> and even if we don't split, we'll still get perf boosts
2008-09-12 22:26 <MaZe> (don't split on :)
2008-09-12 22:26 <MaZe> that's a colon )
2008-09-12 22:26 <flips> I thought it was a smile :)
2008-09-12 22:26 <MaZe> well it was a ':' than a ')'
2008-09-12 22:26 <MaZe> ;-)
2008-09-12 22:27 <flips> ok, immediate goal is to fix the brain damage in my refcounting
2008-09-12 22:27 <MaZe> if you have a dentry and inode already in memory
2008-09-12 22:27 <flips> sorry about the pile of poo I posed ;)
2008-09-12 22:27 <MaZe> how long does it take to fetch the xattrs for that inode?
2008-09-12 22:27 <MaZe> order of magnitude
2008-09-12 22:27 <flips> oh that's another thing... we can easily put a small hash in front of the atom dict
2008-09-12 22:28 <flips> very easily
2008-09-12 22:28 <flips> sub microsecond
2008-09-12 22:28 <flips> I guess
2008-09-12 22:28 <MaZe> [because with the correct implementation the above is less than 50 cycles]
2008-09-12 22:28 <flips> sure, and a microsecond is about 3,000
2008-09-12 22:28 <flips> 60 times slower
2008-09-12 22:29 <flips> so there's something to be gained
2008-09-12 22:29 <flips> I thinjk we gain most of it from putting a hash in front of the dirops
2008-09-12 22:29 <MaZe> hash of what?
2008-09-12 22:29 <flips> so we end up with level 1, level 2
2008-09-12 22:29 <flips> hash of the thing we're atomizing
2008-09-12 22:29 <flips> just keep the common ones there
2008-09-12 22:29 <flips> let the cold ones drop off
2008-09-12 22:30 <MaZe> ok, now you've gotten ahead of me...
2008-09-12 22:30 <flips> it's a linux meme
2008-09-12 22:30 <flips> dentry hash as an example of that
2008-09-12 22:30 <MaZe> ok, how about, first a question: what exactly is an atom (example?)
2008-09-12 22:30 <flips> sucky example
2008-09-12 22:30 <MaZe> [how big is an atom]
2008-09-12 22:30 <flips> it's just a small integer with a name
2008-09-12 22:30 <flips> and a refcount
2008-09-12 22:30 <MaZe> ok, the names, can we have an example?
2008-09-12 22:30 <flips> the name of an atom is up to 255 chars (tradition)
2008-09-12 22:31 <flips> names are unrestricted
2008-09-12 22:31 <MaZe> pascal strings ;-)
2008-09-12 22:31 <flips> right
2008-09-12 22:31 <flips> in fact they are pascal strings
2008-09-12 22:31 <flips> that's what ext2 uses
2008-09-12 22:31 <flips> and what I always use
2008-09-12 22:31 <MaZe> ok, so how would you atomize (where would the atom boundaries be) in the above 3 line xattr example I posted?
2008-09-12 22:32 <flips> beats the crap out of shitty C strings
2008-09-12 22:32 <MaZe> strlen is fast ;-)
2008-09-12 22:32 <flips> sucks compared to looking up a byte
2008-09-12 22:32 <flips> strlen does cacheline damage
2008-09-12 22:32 <flips> "considered harmful to cache lines"
2008-09-12 22:32 <MaZe> I meant strlen is fast on pascal strings
2008-09-12 22:32 <flips> right
2008-09-12 22:33 <flips> I'd atomize the whole 3 line xattr
2008-09-12 22:33 <flips> and store the atom
2008-09-12 22:33 <MaZe> the whole thing?
2008-09-12 22:33 <flips> and have a limit of 2^48 atoms
2008-09-12 22:33 <MaZe> ok, on my drive, you'd have all refcounts = 1
2008-09-12 22:33 <flips> right now, it's 2^32 atoms
2008-09-12 22:33 <flips> if we do bodies, might want to widen that
2008-09-12 22:33 <flips> sure
2008-09-12 22:33 <flips> who cares
2008-09-12 22:34 <MaZe> selinux does 
2008-09-12 22:34 <flips> refcounts take hardly any space
2008-09-12 22:34 <flips> 2 bytes each
2008-09-12 22:34 <flips> now, as soon as something _does_ collide, you know right away
2008-09-12 22:34 <flips> that is
2008-09-12 22:34 <flips> match another body
2008-09-12 22:34 <flips> well
2008-09-12 22:34 <flips> anyway
2008-09-12 22:34 <flips> it's premature
2008-09-12 22:35 <flips> xattrs have to work
2008-09-12 22:35 <flips> or nobody cares how well we compress acls
2008-09-12 22:35 <MaZe> yes, but they have to be treated seperately
2008-09-12 22:35 <MaZe> here - I'll write up how it should be done IMHO
2008-09-12 22:35 <flips> I know, you want to tokenize
2008-09-12 22:35 <flips> the xattr string
2008-09-12 22:35 <flips> and compress that way
2008-09-12 22:36 <flips> so why don't they tokenize?
2008-09-12 22:36 <flips> yes please
2008-09-12 22:36 <flips> and let's invite the selinuxen to read that post
2008-09-12 22:37 <MaZe> builtin:[security.selinux] sedict1:[unconfined_u] sedict2:[object_r] sedict3:[default_t] sedict4:[s0]
2008-09-12 22:37 <MaZe> builtin:[system.posix_acl_access] acl_dict:[0sAgAAAAEABwD/////AgAEAGQAAAAEAAUA/////xAABQD/////IAAFAP////8]
2008-09-12 22:37 <MaZe> builtin:[user.] user_dict:[hash] user_dict:[sdfsdfjhsdjfhsdjkfhdjskahfjkdsahkj]
2008-09-12 22:37 <MaZe> I'm still researching the subject
2008-09-12 22:37 <MaZe> and then you want to store the first two directly within the inode
2008-09-12 22:37 <flips> got to fix my mess here now
2008-09-12 22:37 <flips> that's for one file?
2008-09-12 22:38 <MaZe> sedict1=2=3=4 could be the same dict, potentially could be the same dict as the acl_dict, potentially the same as builtin
2008-09-12 22:38 <MaZe> yes
2008-09-12 22:38 <flips> we stoe all of the directly in the inode
2008-09-12 22:38 <flips> on disk
2008-09-12 22:38 <flips> and cache them in memory
2008-09-12 22:38 <flips> when the inode is loaded
2008-09-12 22:38 <MaZe> so what's the size of an inode on disk?
2008-09-12 22:38 <flips> variable
2008-09-12 22:38 <MaZe> maximum?
2008-09-12 22:39 <flips> from about 40 bytes to unlimited
2008-09-12 22:39 <flips> current limitation is an inode table block
2008-09-12 22:39 <flips> but that will go away
2008-09-12 22:40 <MaZe> hmm, I need to start writing a junk fs
2008-09-12 22:40 <MaZe> to get a better feeling for the kernel interfaces
2008-09-12 22:40 <flips> you'll get an excellent chance very soon
2008-09-12 22:40 <flips> we're going to start by porting a junk fs to kernel
2008-09-12 22:40 <flips> next tuesday we'll do that
2008-09-12 22:41 <MaZe> really?
2008-09-12 22:41 <MaZe> hmm
2008-09-12 22:41 <flips> promise
2008-09-12 22:41 <flips> you said you wanted me to pick up the pace
2008-09-12 22:41 <flips> pick it up we shall
2008-09-12 22:41 <flips> hope we don't lose anybody ;)
2008-09-12 22:41 <MaZe> I'm beginning to think an fs should actually have (at least) two layers
2008-09-12 22:42 <MaZe> basically the frontend (UI / interface with vfs) and the backend (interface with block devices)
2008-09-12 22:42 <MaZe> with the ability for the middle to be network seperated
2008-09-12 22:42 <flips> what about the inodes in the middle?
2008-09-12 22:42 <MaZe> you need a clean api in the middle that deals correctly with coherency issues
2008-09-12 22:43 <MaZe> but I think this is the only way to get a well performing net fs
2008-09-12 22:43 <flips> you are entirely correct, and that is how tux3 is structured
2008-09-12 22:43 <flips> it has the cache level and the block level
2008-09-12 22:43 <flips> they are separately cleanly... better be
2008-09-12 22:43 <flips> or it simply won't work
2008-09-12 22:44 <MaZe> the backend is then a get/set/lock/unlock/notify system
2008-09-12 22:44 <flips> well
2008-09-12 22:44 <flips> kinda
2008-09-12 22:44 <flips> the back end is more like async messages
2008-09-12 22:44 <flips> loosely
2008-09-12 22:44 <flips> very loosely
2008-09-12 22:44 <MaZe> yeah, that kind of describes what I'm thinking
2008-09-12 22:45 <MaZe> hard to phrase really
2008-09-12 22:45 <MaZe> especially since it's still unclear to me ;-)
2008-09-12 22:45 <MaZe> the fact it has to be implemented as to separate pieces is now clear to me, with an interface layer that uses tcp-ip
2008-09-12 22:45 <MaZe> BUT
2008-09-12 22:45 <MaZe> can short-circuit the network stack on local host
2008-09-12 22:46 <flips> great, it was never clear to me ;)
2008-09-12 22:46 <flips> it just came out like that
2008-09-12 22:46 <flips> did it itself
2008-09-12 22:46 <MaZe> here's an example:
2008-09-12 22:47 <MaZe> application -> user space -> kernel space -> vfs layer -> client file system layer -> send rpc call -> network stack -> receive rpc call -> dispatch -> server file system layer -> block device layer
2008-09-12 22:47 <flips> and that's only half the loop
2008-09-12 22:47 <flips> for an nfs
2008-09-12 22:47 <MaZe> now if the tcp/ip network stack layer does cookies or UUID to identify that it's talking to itself, than it can zip it up to
2008-09-12 22:48 <MaZe> client fs layer -> direct dispatch -> server fs layer
2008-09-12 22:48 <MaZe> and of course there's the return path
2008-09-12 22:48 <flips> that's the kind of thinking that originally lead to nfs
2008-09-12 22:48 <flips> actually, the reverse of that
2008-09-12 22:48 <MaZe> and it has to be part sync, part async, part notify
2008-09-12 22:48 <flips> we had your second one
2008-09-12 22:48 <flips> and some genius decided it could easily be hacked to be the first one
2008-09-12 22:48 <flips> anyway
2008-09-12 22:49 <flips> it's not a NFS
2008-09-12 22:49 <MaZe> the real problem is now how to minimize the latency and data sent across the net in the middle
2008-09-12 22:49 <flips> it will become a cluster fs before it becomes an nfs
2008-09-12 22:49 <MaZe> and that relies on doing cache coherency and read/write (various types of) and notifications of changes/lock/lock-breaking correctly
2008-09-12 22:49 <flips> yes
2008-09-12 22:50 <MaZe> anyway... enough about my plans to conquer the world
2008-09-12 22:50 <flips> which _nobody_ in the oss world has succeed in doing well
2008-09-12 22:50 <flips> probably also not in the propietary world either
2008-09-12 22:50 <MaZe> anyway, you need to plan the entire fs from the ground up with the assumption all the clients (even the local host) are remote
2008-09-12 22:50 <flips> since we can't see the code or try it we don't know
2008-09-12 22:50 <MaZe> that way you don't need to deal with the local host specially
2008-09-12 22:50 <flips> well
2008-09-12 22:51 <flips> you don't have to put in remote hooks from the beginning
2008-09-12 22:51 <MaZe> [except for the dispatch optimization]
2008-09-12 22:51 <flips> you just have to be aware of where problems can be created
2008-09-12 22:51 <MaZe> you have to design it as if they were there
2008-09-12 22:51 <MaZe> you might not code it quite like that
2008-09-12 22:51 <MaZe> although I think you should
2008-09-12 22:51 <flips> whre it costs nothing, yes
2008-09-12 22:51 <MaZe> even if the net code is a shim .h file
2008-09-12 22:51 <flips> that's seldom the case
2008-09-12 22:52 <flips> but the real problem is, you're answering a demand that doesn't exist
2008-09-12 22:52 <MaZe> people have been optimizing for the wrong situation ;-)
2008-09-12 22:52 <flips> you're hoping your nfs will be so much more amazing, everybody will use it instead of sucky nfs
2008-09-12 22:52 <MaZe> what demand do you think that is?
2008-09-12 22:52 <flips> but you're likely to be amazed and disappointed
2008-09-12 22:52 <MaZe> truthfully?
2008-09-12 22:52 <flips> there's very little demand for a good nfs outside of hpc
2008-09-12 22:52 <flips> and they like lustre
2008-09-12 22:52 <MaZe> I don't care about who uses it or not ;-)
2008-09-12 22:53 <flips> prefectly happy
2008-09-12 22:53 <flips> they just want it to be more reliable and faster
2008-09-12 22:53 <MaZe> I just like good design
2008-09-12 22:53 <flips> well
2008-09-12 22:53 <flips> just writing a dlm to support it will keep you busy for months
2008-09-12 22:53 <flips> if you know _exactly_ what to do
2008-09-12 22:53 <MaZe> yeah, I know, not a good way to design it ;-)
2008-09-12 22:53 <MaZe> if you want to make money
2008-09-12 22:53 <MaZe> but oh well
2008-09-12 22:54 <flips> you can do it
2008-09-12 22:54 <flips> if you already have something working
2008-09-12 22:54 <flips> that people want
2008-09-12 22:54 <flips> and are willing to bribe you to make even more like what they want
2008-09-12 22:54 <MaZe> eh, dlm's aren't that hard if you have a clean api and don't have to deal with prior borkage
2008-09-12 22:54 <flips> well
2008-09-12 22:54 <flips> "not hard" translates into several months, trust me
2008-09-12 22:54 <MaZe> problem is the leakage of breakage from outside
2008-09-12 22:54 <flips> but prove me wrong
2008-09-12 22:55 <flips> I would like to have a good dlm
2008-09-12 22:55 <flips> in fact
2008-09-12 22:55 <flips> would you be kind enough to post a design note on dlm?
2008-09-12 22:55 <flips> because I'd like to cluster tux3
2008-09-12 22:55 <MaZe> nope, because I don't have the design yet ;-)
2008-09-12 22:55 <flips> by this time next year
2008-09-12 22:55 <flips> well
2008-09-12 22:55 <flips> when?
2008-09-12 22:55 <MaZe> I think I'm going to try writing a junk fs this weekend
2008-09-12 22:55 <MaZe> unless stuff burns (I'm oncall)
2008-09-12 22:55 <flips> great
2008-09-12 22:56 <MaZe> and we'll see how well I understand the kernel apis
2008-09-12 22:56 <flips> I'll check in with you on saturday when you're 50% done
2008-09-12 22:56 <flips> you'll figure them out fast
2008-09-12 22:56 <MaZe> and I'm going to start writing directly in kernel space, because that's the entire purpose of the exercise ;-)
2008-09-12 22:56 <flips> little painful to get some of the crap to behave
2008-09-12 22:56 <MaZe> I believe debugging is probably easiest in kvm?
2008-09-12 22:56 <flips> uml
2008-09-12 22:56 <flips> far and away
2008-09-12 22:56 <MaZe> why?
2008-09-12 22:57 <MaZe> can you strace ?
2008-09-12 22:57 <flips> just: make defconfig ARCH=um && make linux ARCH=um; ./linux ubd0=/my/rootfs
2008-09-12 22:57 <flips> that's it
2008-09-12 22:57 <flips> you can gdb it
2008-09-12 22:57 <MaZe> ah
2008-09-12 22:57 <flips> takes a little coaxing
2008-09-12 22:58 <MaZe> where are the logs stored for this channel btw?
2008-09-12 22:58 <MaZe> checking tux3.org
2008-09-12 22:58 <flips> linked from shapor's page I think, which is linked from tux3.org
2008-09-12 22:58 <MaZe> http://shapor.com/tux3/irclogs/current.txt
2008-09-12 22:58 <MaZe> hehe, uptodate to the second
2008-09-12 22:59 <flips> ok, you will be having fun writing lots of new fs code and I will be slaving away finishing xattrs
2008-09-12 22:59 <flips> you got the better deal
2008-09-12 23:00 <MaZe> $ wget -q -O - "http://shapor.com/tux3/irclogs/current.txt" | cut -b18- | sed -rn 's@^<([^>]*)>.*@\1@p' | sort | uniq -c | sort -nr | head -n 9
2008-09-12 23:00 <MaZe>    6427 flips
2008-09-12 23:00 <MaZe>    1199 shapor
2008-09-12 23:00 <MaZe>     908 MaZe
2008-09-12 23:00 <MaZe>     818 bh
2008-09-12 23:00 <MaZe>     616 konrad
2008-09-12 23:00 <MaZe>     369 tim_dimm
2008-09-12 23:00 <MaZe>     113 vandenoever
2008-09-12 23:00 <MaZe>     104 RazvanM
2008-09-12 23:00 <MaZe>      96 flipz
2008-09-12 23:00 <MaZe> interesting stats there
2008-09-12 23:01 <MaZe> how the hell I'm number 3 on that list I'll never know...
2008-09-12 23:01 <flips> you're moving up fast
2008-09-12 23:01 <flips> fast typer
2008-09-12 23:01 <konrad> just have to press the enter key enough :)
2008-09-12 23:01 <MaZe> well, yeah
2008-09-12 23:01 <flips> and wiggle those fingers
2008-09-12 23:01 <MaZe> now everybody is just typing stuff to boost their ranking ;-)
2008-09-12 23:02 <flips> nice example of sed chickentracks
2008-09-12 23:02 <MaZe> I am a sed-maniac
2008-09-12 23:02 <flips> you can cut and paste your code examples if you need a quick boost
2008-09-12 23:03 <MaZe> right, well, I also have a copy of spore waiting for me...
2008-09-12 23:03 <MaZe> wonder if it's any good
2008-09-12 23:04 <flips> let me know
2008-09-12 23:04 <MaZe> and I should reinstall my desktop with the next version of ubuntu
2008-09-12 23:04 <flips> my 4 year old can't wait to get her hands on pure
2008-09-12 23:04 <flips> the quad racing game
2008-09-12 23:04 <flips> from disney
2008-09-12 23:04 <flips> demon is much fun
2008-09-12 23:04 <flips> demo
2008-09-12 23:05 <flips> I'll probably take a few spins around the italian track after I do the next refcount iter
2008-09-12 23:05 <bh> folks
2008-09-12 23:05 <flips> 		unsigned attomoff = (atom << 1) & (-1 << blockbits);
2008-09-12 23:05 <flips> hey bh
2008-09-12 23:05 <MaZe> re above: new fs code includes xattrs eventually ;-)
2008-09-12 23:05 <flips> wow, when I need a reason not to code, one quickly arrives
2008-09-12 23:05 <flips> good luck with that ;)
2008-09-12 23:06 <flips> for me, a week on xattrs alone
2008-09-12 23:06 <flips> maybe you're faster
2008-09-12 23:06 <MaZe> uhm that code you posted looks wrong
2008-09-12 23:06 <MaZe> missing ~
2008-09-12 23:06 <flips> yeah
2008-09-12 23:06 <MaZe> unsigned attomoff = (atom << 1) & ~(-1 << blockbits);
2008-09-12 23:06 <flips> that's why I pasted it ;)
2008-09-12 23:06 <flips> better than a compiler
2008-09-12 23:06 <MaZe> oh, sorry
2008-09-12 23:06 <MaZe> didn't realize it was a quiz
2008-09-12 23:06 <flips> heh
2008-09-12 23:07 <flips> no it was me actually fucking up
2008-09-12 23:07 <flips> in a way
2008-09-12 23:07 <MaZe> oh paste in wrong window?
2008-09-12 23:07 <flips> and in a way, not
2008-09-12 23:07 <flips> no
2008-09-12 23:07 <flips> I pasted it, you saw the bug
2008-09-12 23:07 <flips> nice
2008-09-12 23:07 <MaZe> heh
2008-09-12 23:08 <flips> 		unsigned block = ATOM_REFCOUNT_BLOCK + ((atom >> blockits) << 1));
2008-09-12 23:09 <MaZe> unsigned block = ATOM_REFCOUNT_BLOCK + (atom >> (blockits - 1));
2008-09-12 23:09 <MaZe> the above is wrong
2008-09-12 23:09 <MaZe> again ;-)
2008-09-12 23:09 <MaZe> since it's always even
2008-09-12 23:09 <MaZe> are you running an IQ test or something?
2008-09-12 23:10 <flips> a stupidity test on myself
2008-09-12 23:10 <flips> atom is not always even
2008-09-12 23:10 <flips> why would it be?
2008-09-12 23:10 <MaZe> block is
2008-09-12 23:11 <flips> ((atom >> blockits) << 1)) <- always even
2008-09-12 23:11 <MaZe> the code you pasted results in blocks even/oddness being constant
2008-09-12 23:11 <flips> yeah, gosh
2008-09-12 23:11 <MaZe> I'd assume you don't want that
2008-09-12 23:11 <flips> well actually I think I do
2008-09-12 23:11 <flips> the even block is for the low 16 bits
2008-09-12 23:11 <flips> the odd for the high
2008-09-12 23:12 <MaZe> then you're still 1 off
2008-09-12 23:12 <flips> probably
2008-09-12 23:12 <MaZe> since what you described is true with double 8 bits
2008-09-12 23:12 <MaZe> not with double 16 bits
2008-09-12 23:12 <flips> I was definitely conflating things and being fuzzy
2008-09-12 23:12 <flips> that's why you will get your xattrs written in _less_ than a week
2008-09-12 23:13 <MaZe> unsigned block = ATOM_REFCOUNT_BLOCK + (atom >> (blockits - 1)) << 1;
2008-09-12 23:13 <MaZe> is what you want if you want double-blocks of u16s with low and high blocks
2008-09-12 23:13 <flips> yes
2008-09-12 23:13 <flips> thanks
2008-09-12 23:13 <MaZe> although that deserves a comment ;-)
2008-09-12 23:13 <flips> see, now I'm getting semantically addressable code
2008-09-12 23:14 <flips> I loosely indicate the semantics, and the code comes back
2008-09-12 23:14 <MaZe> what do you mean?
2008-09-12 23:14 <flips> that's the one we want I think
2008-09-12 23:14 <MaZe> I think you'd actually want to spread it in a different way for performance reasons
2008-09-12 23:15 <MaZe> first all the low blocks, then all the high blocks
2008-09-12 23:15 <MaZe> since the high blocks will almost never be updated
2008-09-12 23:15 <MaZe> thus you want low blocks to be sequential on disk
2008-09-12 23:15 <flips> 16 bit count with carry into the high block is a very nice balance between getting lots of atoms onto a block and not carrying too often
2008-09-12 23:15 <flips> there's no advantage to separating it that way
2008-09-12 23:15 <flips> that I know of
2008-09-12 23:15 <flips> a _very_ small advantage in the radix tree lookup
2008-09-12 23:16 <flips> but it was my first idea, what you said
2008-09-12 23:16 <flips> and I might stick with that indeed
2008-09-12 23:16 <MaZe> I think having two different ATOM_REFCOUNT_BLOCK would make for cleaner easier to understand code
2008-09-12 23:16 <flips> two different?
2008-09-12 23:16 <MaZe> ATOM_REFCOUNT_{LOW,HIGH}_BLOCK
2008-09-12 23:17 <flips> sure
2008-09-12 23:17 <flips> it's that way now
2008-09-12 23:17 <flips> before you posted that expression
2008-09-12 23:17 <MaZe> I haven't actually looked at the code ;-)
2008-09-12 23:17 <flips> just buggy
2008-09-12 23:17 <flips> ok
2008-09-12 23:17 <flips> give me an hour
2008-09-12 23:17 <flips> and you get to look at working, tested code
2008-09-12 23:17 <flips> I program much better when I'm not chatting
2008-09-12 23:18 <MaZe> ok, I think I'm gonna finally head home from work, and maybe reboot into mac os x and start up spore - and stop bothering you ;-)
2008-09-12 23:18 <flips> wow
2008-09-12 23:18 <flips> didn't realize you're camping out there
2008-09-12 23:18 <flips> but why not
2008-09-12 23:18 <flips> infinite junk food
2008-09-12 23:18 <flips> sweet sound of vacuum cleaners in the distance
2008-09-12 23:19 <MaZe> plus 30" screen
2008-09-12 23:19 <MaZe> not to mention my place is a studio
2008-09-12 23:19 <flips> that can be fixed
2008-09-12 23:19 <flips> just put in a ticket
2008-09-12 23:19 <MaZe> has an empty fridge (ok, I have spaghetti)
2008-09-12 23:19 <flips> you'll have one at home
2008-09-12 23:19 <flips> yah
2008-09-12 23:19 <MaZe> is a total mess
2008-09-12 23:19 <flips> you need to get that ramyun
2008-09-12 23:20 <flips> get it on the way home
2008-09-12 23:20 <MaZe> and is lacking (even after more than 2 years) basic amenities like a desk
2008-09-12 23:20 <flips> ouch
2008-09-12 23:20 <flips> just order one online
2008-09-12 23:20 <flips> ikea
2008-09-12 23:20 <flips> have it in 3 days
2008-09-12 23:20 <MaZe> because I've never found a good way to fit one in
2008-09-12 23:20 <flips> some assembly required
2008-09-12 23:20 <flips> that's the trick
2008-09-12 23:20 <MaZe> so to be fair, even back in Poland, when I had a desk, I spent most time with laptop on lap on bed
2008-09-12 23:20 <flips> and move to phoenix where you can have a real house ;)
2008-09-12 23:20 <MaZe> which is one of the reasons I've never bothered
2008-09-12 23:21 <flips> how do you say "hello" in polish?
2008-09-12 23:21 <flips> hallo?
2008-09-12 23:21 <MaZe> Dzień Dobry.
2008-09-12 23:21 <MaZe> is Good Day
2008-09-12 23:21 <MaZe> hello is 'halo'
2008-09-12 23:21 <flips> characters didn't work in xchat
2008-09-12 23:21 <MaZe> but that's kind of like pulled in from english
2008-09-12 23:22 <flips> that's the unaccented version of above?
2008-09-12 23:22 <MaZe> Dzien' Dobry
2008-09-12 23:22 <MaZe> n with accent like /
2008-09-12 23:22 <flips> got it
2008-09-12 23:22 <flips> like sheen dobry
2008-09-12 23:22 <flips> kinda
2008-09-12 23:22 <flips> with more bite
2008-09-12 23:22 <MaZe> and 'halo' is more like a phone greeting when you pick up then really hello
2008-09-12 23:23 <flips> ok, I'll try it on a pole tomorrow and see if it works
2008-09-12 23:23 <MaZe> you're more likely to use 'hej' (hey) between friends in person, and dzien dobry for more formal purposes and 'halo' when picking up the phone
2008-09-12 23:23 <MaZe> hmm
2008-09-12 23:23 <MaZe> let me write it down more phonetically
2008-09-12 23:23 <MaZe> ugh
2008-09-12 23:24 <MaZe> dobry = [hard d] [short o] [hard b] [hard trilled r] [short i/y]
2008-09-12 23:25 <flips> oh yeah I can say dobry
2008-09-12 23:25 <flips> even with the trill
2008-09-12 23:25 <MaZe> so dzien, is dzie - en - two syllables
2008-09-12 23:26 <flips> tshee - ehn
2008-09-12 23:26 <flips> ?
2008-09-12 23:26 <MaZe> were dzi is a consonant,  e is a vowel and n' is a soft nasal n
2008-09-12 23:26 <flips> nice
2008-09-12 23:26 <MaZe> okay maybe more like one syllable, kind of hard to say because soft vowels (and there's two of them here) are kind of syllable like
2008-09-12 23:27 <MaZe> so the i in dzi takes the sound dz and makes it soft
2008-09-12 23:27 <flips> good enough to try
2008-09-12 23:27 <flips> I'll get corrected soon enough
2008-09-12 23:27 <MaZe> which is exactly the purpose of the accent on the n (accent on consonants is written as an i infront of a vowel - hence the dz "i" en'
2008-09-12 23:27 <MaZe> that's really dz'en'
2008-09-12 23:28 <flips> ah
2008-09-12 23:28 <MaZe> and dz is a single letter
2008-09-12 23:28 <flips> wow, more complex than say czech
2008-09-12 23:28 <MaZe> that just happens to be written with two
2008-09-12 23:28 <MaZe> since we ran out of latin characters
2008-09-12 23:29 <MaZe> hence dz rz sz cz ch  should basically be treated as single letters
2008-09-12 23:29 <MaZe> and then there's sounds like drz brz and so on, which are almost like a single letter
2008-09-12 23:29 <flips> ugh
2008-09-12 23:29 <flips> with a trill?
2008-09-12 23:30 <MaZe> while you can theoretically read it as b-rz and d-rz, almost everybody pronounces is it quickly where it kind of melds into one
2008-09-12 23:30 <MaZe> and there's no 'r' in their ;-)
2008-09-12 23:30 <MaZe> since rz is actually 'z with a dot'  ie. ż
2008-09-12 23:30 <MaZe> which is the first letter of my last name
2008-09-12 23:30 <MaZe> which is almost like the j in french 'je'
2008-09-12 23:30 <MaZe> so more like 'zh'
2008-09-12 23:31 <flips> IC
2008-09-12 23:31 <MaZe> it's actually very consistant
2008-09-12 23:31 <MaZe> very few words can't be read correctly if you know the relatively short set of rules
2008-09-12 23:31 <MaZe> there are very few exceptions
2008-09-12 23:31 <flips> I'll learn those rules
2008-09-12 23:31 <flips> over time
2008-09-12 23:32 <flips> slavic languages are fun
2008-09-12 23:32 <flips> great langues for being cynical in
2008-09-12 23:32 <flips> from what I have seen
2008-09-12 23:32 <MaZe> an example being frozen (zmarznięty), where the syllable split is zmar-znię-ty  [the ę is nasal e, or e with a , ogonek accent - like french c with cedilla, but flipped other way]
2008-09-12 23:32 <flips> whoops
2008-09-12 23:32 <flips> none of those chars are working on braindead xchat
2008-09-12 23:32 <MaZe> so even though you see rz it ain't zh
2008-09-12 23:33 <MaZe> those were all the same character
2008-09-12 23:33 <flips> time for a less braindead chat client
2008-09-12 23:33 <MaZe> probably because I'm writing in utf8
2008-09-12 23:33 <flips> yes
2008-09-12 23:33 <MaZe> although they show up fine in pidgin
2008-09-12 23:33 <flips> but xchat should grok that
2008-09-12 23:33 <MaZe> xchat probably does
2008-09-12 23:33 <flips> xchat sucks at everything by default
2008-09-12 23:33 <MaZe> I'd guess your terminal is not utf8 or something
2008-09-12 23:33 <flips> I wouldn't assume taht
2008-09-12 23:33 <flips> it doesn't run in a terminal
2008-09-12 23:35 <MaZe> oh, it doesn't?
2008-09-12 23:35 <MaZe> oh right the x ;-)
2008-09-12 23:36 <MaZe> what are we on here?
2008-09-12 23:36 -!- maze_pallas(~elbereth@216-239-45-4.google.com) has joined #tux3
2008-09-12 23:36 <MaZe> oftc
2008-09-12 23:36 <MaZe> let's see if my xchat works
2008-09-12 23:36 <MaZe> ąćęłńóśźż
2008-09-12 23:36 <MaZe> works 4 me
2008-09-12 23:36 <maze_pallas> ąćęłńóśźż
2008-09-12 23:36 <maze_pallas> yup
2008-09-12 23:37 <maze_pallas> ĄĆĘŁŃÓŚŹŻ
2008-09-12 23:37 <konrad> xchat does utf8 great here
2008-09-12 23:37 <MaZe> ĄĆĘŁŃÓŚŹŻ
2008-09-12 23:37 <flips> well
2008-09-12 23:37 <MaZe> right
2008-09-12 23:37 <flips> I have to set something
2008-09-12 23:37 <flips> or upgrade
2008-09-12 23:37 <MaZe> but you probably need to have proper locale
2008-09-12 23:37 <flips> oh
2008-09-12 23:37 <MaZe> xchat_2.6.1-0ubuntu2_i386.deb here
2008-09-12 23:37 <konrad> I have xchat 2.8.4
2008-09-12 23:37 <MaZe> freshly installed
2008-09-12 23:37 <flips> I thought unicode was supposed to be independent of locale
2008-09-12 23:38 -!- maze_(~maze@216-239-45-4.google.com) has joined #tux3
2008-09-12 23:38 <maze_> testing ;-)
2008-09-12 23:38 <maze_> ?????�???
2008-09-12 23:38 <maze_> nope
2008-09-12 23:38 <flips> 2.6.8-0.3
2008-09-12 23:38 <flips> etch
2008-09-12 23:38 <flips> :p
2008-09-12 23:38 <MaZe> try LC_ALL=en_US.utf-8 xchat as the startup command
2008-09-12 23:39 -!- maze__(~maze@216-239-45-4.google.com) has joined #tux3
2008-09-12 23:39 <maze__> testing
2008-09-12 23:39 <maze__> ĄĆĘŁŃÓŚŹŻ ąćęłńóśźż
2008-09-12 23:39 <maze__> yep works
2008-09-12 23:39 <MaZe> it's gotten crowded in here
2008-09-12 23:40 <MaZe> I guess I don't really know what that means...
2008-09-12 23:40 <flips> amazed at all the maze clones
2008-09-12 23:40 <flips> you better get home
2008-09-12 23:40 <flips> I need to hear about spore
2008-09-12 23:40 <MaZe> but obviously something is broken if xchat's wire encoding depends on the locale
2008-09-12 23:41 <MaZe> maybe there's a switch or something
2008-09-12 23:42 <flips> I'm giving up on getting xchat to show those chars
2008-09-12 23:42 <MaZe> ah no idea
2008-09-12 23:42 <MaZe> just quit
2008-09-12 23:42 <flips> one day I will do a gui for irssi
2008-09-12 23:42 <MaZe> and restart with LC_ALL=en_US.utf-8 xchat
2008-09-12 23:42 <flips> keep promising myself
2008-09-12 23:42 <MaZe> decloned.
2008-09-12 23:43 <MaZe> going to try the above? or shall I head home?
2008-09-12 23:44 <flips> head home
2008-09-12 23:44 <MaZe> ok will do so then
2008-09-12 23:44 <flips> I'll have it working when you get there
2008-09-12 23:44 <flips> and you can start your fs
2008-09-12 23:44 <flips> or your spore review
2008-09-12 23:45 <flips> pick up some ramyun on the way in case it turns into a long one
2008-09-12 23:45 <MaZe> ok, might do so, there's a store on route after all...
2008-09-12 23:48 <bh> hey
2008-09-12 23:48 <flips> hi
2008-09-12 23:48 <bh> yeah, just got some crude gdb script to scan through all system threads and print out their state
2008-09-12 23:48 <bh> more to come
2008-09-12 23:49 <bh> this is to help with core examinations
2008-09-12 23:49 <bh> which it seems that nobody does under Linux
2008-09-12 23:49 <flips> true
2008-09-12 23:49 <flips> I have never
2008-09-12 23:49 <flips> should
2008-09-12 23:50 <bh> it's not a criticism, just a new tool to help with debugging
2008-09-12 23:50 <bh> I'm getting a bunch of nfsd threads in state 4 which is a bit odd
2008-09-12 23:51 <flips> it's about time I learned summa that fu
2008-09-12 23:53 <bh> summa ?
2008-09-12 23:53 <bh> =some of
2008-09-12 23:53 <bh> ok
2008-09-12 23:53 <flips> yap
2008-09-12 23:53 <bh> that's all we did at netapp practically
2008-09-12 23:53 <bh> for better or worse
2008-09-12 23:53 <flips> gotta get me summa data
2008-09-12 23:53 <flips> gotta get me summa dat