2009-01-23 00:11 -!- RazvanM(~RazvanM@96.234.239.248) has joined #tux3 2009-01-23 08:43 -!- tim_dimm(~timothyhu@cpe-76-168-94-231.socal.res.rr.com) has joined #tux3 2009-01-23 09:03 -!- pixelbeat(~padraig@84.203.137.218) has joined #tux3 2009-01-23 09:19 -!- tim_dimm(~timothyhu@cpe-76-168-94-231.socal.res.rr.com) has joined #tux3 2009-01-23 09:26 -!- kushal(~kushal@115.109.15.140) has joined #tux3 2009-01-23 09:59 -!- mingming(~mingming@32.97.110.51) has joined #tux3 2009-01-23 10:04 -!- tim_dimm_(~timothyhu@cpe-76-168-94-231.socal.res.rr.com) has joined #tux3 2009-01-23 10:06 -!- tim_dimm(~timothyhu@cpe-76-168-94-231.socal.res.rr.com) has joined #tux3 2009-01-23 10:29 -!- amey(~amey@115.109.15.140) has joined #tux3 2009-01-23 10:39 -!- gaurav(~gaurav@115.109.15.140) has joined #tux3 2009-01-23 10:39 -!- kushal(~kushal@115.109.15.140) has joined #tux3 2009-01-23 11:31 -!- RazvanM(~RazvanM@dazzler.isi.jhu.edu) has joined #tux3 2009-01-23 11:41 -!- dcg(~dcg@125.pool80-103-0.dynamic.orange.es) has joined #tux3 2009-01-23 12:35 -!- kushal_(~kushal@115.109.10.125) has joined #tux3 2009-01-23 13:00 -!- kushal_(~kushal@115.109.10.125) has joined #tux3 2009-01-23 13:22 hirofumi, there? 2009-01-23 13:22 yes 2009-01-23 13:22 I will generalize defer_free to have a struct header instead of all fields in sb 2009-01-23 13:23 because we will have two defer lists, I think 2009-01-23 13:23 it will probably conflict with your struct link cleanup 2009-01-23 13:23 so maybe just put that patch aside for now? 2009-01-23 13:23 no problem, it's very good 2009-01-23 13:23 I was also thinking about it 2009-01-23 13:24 and removing extent_t 2009-01-23 13:24 yes 2009-01-23 13:24 struct defer { struct link *tail; u64 *pos, top; }; 2009-01-23 13:24 for now, we can make it even more general maybe... but that's all we need 2009-01-23 13:25 I think general extent would be struct seg 2009-01-23 13:25 yes 2009-01-23 13:25 it's shorter to type too :) 2009-01-23 13:25 and I think extent_t is just defer-free local structure 2009-01-23 13:25 yes 2009-01-23 13:26 so, I was thinking remove (actually move to local place) it 2009-01-23 13:27 it's not actually used in any code yet 2009-01-23 13:27 but it will be soon 2009-01-23 13:27 so I think it's best to keep it in the tree, it helps us move a little faster 2009-01-23 13:27 Iirc, retire_defree is using 2009-01-23 13:28 right, and retire_defree isn't used yet 2009-01-23 13:28 but soon 2009-01-23 13:29 it was defree, it is using extent_t to store it 2009-01-23 13:29 ah, and retire_defree too 2009-01-23 13:30 the next revision will use u64 instead of extent_t 2009-01-23 13:31 ah, yes 2009-01-23 13:31 I guess actually diskextent? 2009-01-23 13:31 diskextent is be_ 2009-01-23 13:31 yes 2009-01-23 13:31 ah, it's not store to disk 2009-01-23 13:32 right 2009-01-23 13:33 well, u64 or something structure 2009-01-23 13:33 I see three uses for this now: defer frees per delta; defer frees per rollup (metdata redirect); on replay... bookkeeping between the replay passes 2009-01-23 13:34 i see 2009-01-23 13:34 well, I guess exporting defree to other place is not good idea 2009-01-23 13:35 exporting defree internal 2009-01-23 13:35 yes, just internal 2009-01-23 13:35 good 2009-01-23 13:35 it's a highly specific task... keep a list of u64's 2009-01-23 13:36 yes, sounds good 2009-01-23 13:36 -!- gila(~gila@5ED41295.cable.ziggo.nl) has joined #tux3 2009-01-23 13:36 I was thinking about hackfs's io functions 2009-01-23 13:37 what is main purpose for current task? 2009-01-23 13:39 why use them? 2009-01-23 13:39 yes, almost 2009-01-23 13:39 to get them tested... they move towards no block library, which moves towards no buffers 2009-01-23 13:40 and vecio/syncio... moves towards a direct path from vfs IO to bio transfer 2009-01-23 13:40 yes, I guess it's good itself 2009-01-23 13:40 with several layers removed, and get rid of the get_block bottleneck 2009-01-23 13:40 but, I thought we can do after atomic commit 2009-01-23 13:40 possibly 2009-01-23 13:41 i see 2009-01-23 13:41 ok 2009-01-23 13:41 well, I'm going to think about it more 2009-01-23 13:41 for now 2009-01-23 13:41 good 2009-01-23 13:42 I agree with the principle of doing only what we need to do 2009-01-23 13:42 now I found is fork_buffer() can be race on some arch (weak memory order) 2009-01-23 13:43 where is the race? 2009-01-23 13:43 and for write, we will need async and sync version 2009-01-23 13:44 data copy and state is no locking 2009-01-23 13:44 actually, it is using lock_buffer(), but it's to sync with io 2009-01-23 13:45 I think state can be leak to another cpu before data copy 2009-01-23 13:45 I left the locking out because I was not sure exactly what was needed, and our initial usage is synchronous 2009-01-23 13:46 so it could be mb() or spinlock 2009-01-23 13:48 on current locking (e.g. btree->lock), I guess we want smp_wmb() or spinlock logically 2009-01-23 13:48 well, it would be rare, and would not happen on x86 2009-01-23 13:49 asynchronous use of fork_buffer will be some time in the future, so there is time to think about it 2009-01-23 13:49 synchronous use is needed very soon 2009-01-23 13:50 by synchronous, I mean only one task 2009-01-23 13:50 I think read and buffer_fork are not serialized 2009-01-23 13:50 it's protected by delta_lock for now 2009-01-23 13:51 this is very crude 2009-01-23 13:51 well, what I want to say here, I want to share about issue 2009-01-23 13:51 the both of fork_buffer and read have down_read()? 2009-01-23 13:52 for now, fork_buffer will have down_write 2009-01-23 13:53 because it is only used to flush the bitmaps 2009-01-23 13:53 and dirent pages? 2009-01-23 13:53 ah 2009-01-23 13:54 give me 5 minutes to think :) 2009-01-23 13:54 ok :) 2009-01-23 13:58 I think you're right 2009-01-23 13:59 ok 2009-01-23 13:59 ah well, that is a good reason to get the SMP issue correct now :) 2009-01-23 14:00 ah, you are tring to delay about it 2009-01-23 14:00 more like, let it improve over time 2009-01-23 14:00 yes 2009-01-23 14:01 directory change has to be inside change_begin/end, so maybe it is protected by delta_lock 2009-01-23 14:01 but still, it does not hurt to make it fully SMP safe 2009-01-23 14:01 well, smp issue is a bit hard to improve incrementally in my mind 2009-01-23 14:02 I imagined the initial usage would be protected by delta_lock 2009-01-23 14:02 i see 2009-01-23 14:02 I think we can down_write always if we want 2009-01-23 14:03 or mutex instead of rwsem 2009-01-23 14:03 yes 2009-01-23 14:04 well, it would be a bit later 2009-01-23 14:04 I meant I'll think about hackfs for now 2009-01-23 14:05 smp improvement is welcome 2009-01-23 14:06 ok 2009-01-23 14:06 it would be nice to show the fork_buffer idea on lkml 2009-01-23 14:06 yes, I think it's good idea and can be generarize for all fs 2009-01-23 14:07 to avoid needless io wait 2009-01-23 14:08 any time you are ready for a pull, I am ready, I will merge my defer free changes with your link improvements after pull 2009-01-23 14:09 ok 2009-01-23 14:10 I'll prepare 3 patches for link 2009-01-23 14:13 ACTION will be back in 1 hour 2009-01-23 14:13 ok, I'll post those until you back 2009-01-23 15:13 static-http://userweb.kernel.org/~hirofumi/tux3/ 2009-01-23 15:14 not only 3 patches though 2009-01-23 15:34 reading 2009-01-23 15:40 - link_add(page_link(page), sb->defree); 2009-01-23 15:40 - sb->defree = sb->defree->next; 2009-01-23 15:40 + link_add(page_link(page), &sb->defree); <- nice :) 2009-01-23 15:40 thanks 2009-01-23 15:40 it is main purpose of link change 2009-01-23 15:41 it's exactly equivalent, which I didn't see 2009-01-23 15:42 well is it 2009-01-23 15:42 it puts sb.defree on the list as a member 2009-01-23 15:43 yes 2009-01-23 15:44 doesn't that build the queue in the reverse order? 2009-01-23 15:44 with it, it can remove special case 2009-01-23 15:44 -!- gila(~gila@5ED41295.cable.ziggo.nl) has joined #tux3 2009-01-23 15:44 yes 2009-01-23 15:44 it doesn't have the pointer to back 2009-01-23 15:45 one of my goals in this application was to replay the queue in the same order as it is created 2009-01-23 15:46 it doesn't matter for deferred free, but it does matter for the other use I see in replay 2009-01-23 15:46 to get it, I think we have to add the pointer to tail, and link_add_tail would be used for it 2009-01-23 15:46 incidentally, in tux2 I did the same thing as you have written here: stored lists on pages, replayed the pages in reverse order 2009-01-23 15:47 yes 2009-01-23 15:47 or walk until tail to add 2009-01-23 15:48 I think I would like to merge the part of this that doesn't touch log.c for now 2009-01-23 15:49 eh, I thought you said you want it 2009-01-23 15:50 btw, why? 2009-01-23 15:50 both is LIFO order 2009-01-23 15:51 the current one is fifo 2009-01-23 15:51 according to my test 2009-01-23 15:51 oh, let me check it 2009-01-23 15:52 fifo queue with requirement that fifo never has less than one element, which is taken care of by init 2009-01-23 15:53 I can imagine a fifo that removes that special case by putting a dedicated tail element on the list 2009-01-23 15:53 but it needs some fiddling 2009-01-23 15:57 -void change_begin(struct sb *sb) { }; 2009-01-23 15:57 -void change_end(struct sb *sb) { }; 2009-01-23 15:57 +void change_begin(struct sb *sb) { } 2009-01-23 15:57 +void change_end(struct sb *sb) { } 2009-01-23 15:57 <- thanks :) 2009-01-23 15:58 ok, I've got 2009-01-23 15:58 I think we can it with link_add() 2009-01-23 16:03 the way I thought about doing it is, have a dedicated head element, and access the list through a pointer to tail 2009-01-23 16:05 then, insert new element by list_add(tail) and remove from head by list_del_next(tail->next) 2009-01-23 16:06 sorry 2009-01-23 16:06 link_ 2009-01-23 16:09 yes 2009-01-23 16:09 well, so, those patches don't need to merge for now 2009-01-23 16:10 it's fine, I can pull them all 2009-01-23 16:10 as they are 2009-01-23 16:11 and then fiddle with log.c 2009-01-23 16:11 I have a patch in progress to allow two deferred free lists 2009-01-23 16:13 link patch is just broken, if you are ok, please pull 2009-01-23 16:13 I'm ok with it 2009-01-23 16:21 pulled 2009-01-23 16:22 pushed to public 2009-01-23 16:30 hirofumi, it feels like another golden copy 2009-01-23 16:30 well, with delalloc and vol_bread 2009-01-23 16:30 it needs some patches though 2009-01-23 16:31 right 2009-01-23 16:31 any issues? 2009-01-23 16:31 needs patch to kernel 2009-01-23 16:31 that's an issue, what patch? 2009-01-23 16:31 or newer kernel version 2009-01-23 16:31 I vote for newer kernel version 2009-01-23 16:32 yes, it would be more easy 2009-01-23 16:32 ah, and needs temporary patches 2009-01-23 16:32 like mark_buffer_dirty and set_buffer_uptodate 2009-01-23 16:33 temporary patches are ok 2009-01-23 16:33 how do we make things easy for our students doing the dedup work? 2009-01-23 16:33 it is 5:30 am there, so I can't ask them if new kernel is ok, I think it probabably is though 2009-01-23 16:33 it is kernel, so there is not needed those patches 2009-01-23 16:34 ah right 2009-01-23 16:34 ah, if we merge those 2009-01-23 16:34 and I think they will be ok working with more recent kernel 2009-01-23 16:34 when they start working with kernel 2009-01-23 16:34 yes 2009-01-23 16:34 it is probably more interesting for them anyway 2009-01-23 16:35 well, I don't recommend to merge temporary patches like mark_buffer_dirty() and set_buffer_uptodate 2009-01-23 16:35 it just makes happy current flusher 2009-01-23 16:35 for testing purpose 2009-01-23 16:36 happy in what sense? 2009-01-23 16:36 buffer state management for current flusher 2009-01-23 16:37 e.g. mark_buffer_dirty() requires buffer_uptodate state 2009-01-23 16:37 and dirty after modify 2009-01-23 16:38 ok, it sounds good 2009-01-23 16:39 it means reverting your some change for atomic commit 2009-01-23 16:39 with one? 2009-01-23 16:39 the clean vs uptodate? 2009-01-23 16:40 http://userweb.kernel.org/~hirofumi/revert-mark_buffer_dirty.patch 2009-01-23 16:40 the patch is this 2009-01-23 16:40 reintroduce mark_buffer_dirty() after blockdirty() 2009-01-23 16:40 and... 2009-01-23 16:41 don't assume new_block() buffer is dirty 2009-01-23 16:41 that is fine, it's a small local change 2009-01-23 16:41 if those are ok, it would work for new kernel 2009-01-23 16:42 they are ok 2009-01-23 16:43 ok, I'll prepare to merge those, with delalloc and vol_bread 2009-01-23 16:43 yes, here are my three most exciting things right now: delalloc; vol_bread; new kernel 2009-01-23 16:44 -!- tim_dimm(~timothyhu@cpe-76-168-94-231.socal.res.rr.com) has joined #tux3 2009-01-23 16:44 and the 26 hour survival is fine :) 2009-01-23 16:44 ah, btw, without delalloc and vol_bread is necessary to survive 2009-01-23 16:45 requirement is bitmap lock fix and memory allocation change 2009-01-23 16:45 and temporary patche 2009-01-23 16:45 and temporary patches 2009-01-23 16:45 that is ok, it is just interesting to know it survived that long with some minor differences 2009-01-23 16:45 well, probably, any convination of patches 2009-01-23 16:46 it is more interesting to work on stabilizing with delalloc and vol_bread now, those are the things that move us forward 2009-01-23 16:46 the patch depending newer kernel is only delalloc 2009-01-23 16:46 oh :) 2009-01-23 16:47 right, it's just the buffer_delay check 2009-01-23 16:47 well, so we can choise new kernel with delalloc, or current kernel without delalloc 2009-01-23 16:47 yes 2009-01-23 16:47 I vote for new kernel with delalloc 2009-01-23 16:48 ok 2009-01-23 16:48 I'll prepare 2009-01-23 16:48 after sleep 2009-01-23 16:48 ACTION was getting tired of 2.6.26.5 anyway 2009-01-23 16:48 ok 2009-01-23 16:49 I will hack log.c to allow multiple defer_free lists, and start running cursor_redirect today 2009-01-23 16:49 btw, if we use symlink to tux3/user/kernel, we don't need full tree for now 2009-01-23 16:50 full tree? 2009-01-23 16:50 git tree from linus 2009-01-23 16:50 it's it getting huge? 2009-01-23 16:50 it may just be lazy 2009-01-23 16:51 lazy to prepare tree 2009-01-23 16:51 if we merge those patches, current repo our git tree will not work with new hg tree 2009-01-23 16:52 current our git tree 2009-01-23 16:52 so, I thought we may be lazy to prepare git tree for it 2009-01-23 16:53 if not, there is no problem 2009-01-23 16:53 you mean the 2.6.26.5 git tree? 2009-01-23 16:53 yes 2009-01-23 16:53 -!- tim_dimm(~timothyhu@cpe-76-168-94-231.socal.res.rr.com) has joined #tux3 2009-01-23 16:53 I am ready to say goodbyte to 2.6.26.5 completely 2009-01-23 16:53 delalloc requires more newler version 2009-01-23 16:53 yes, it is time to make the move to current linus, and set up on kernel.org 2009-01-23 16:54 say sayonara to 2.6.26.5 2009-01-23 16:54 ok 2009-01-23 16:54 I think we are maybe one week away from atomic commit in userspace and kernel, on current linus tree 2009-01-23 16:55 it is temporary? or will use for long time 2009-01-23 16:55 ? 2009-01-23 16:55 that will be a milestone 2009-01-23 16:55 the linus tree? 2009-01-23 16:55 yes 2009-01-23 16:55 only until we can get a spot in linux-next :) 2009-01-23 16:56 ah, I thought about to make good history on git 2009-01-23 16:56 it would be nice 2009-01-23 16:56 but not necessary 2009-01-23 16:56 if somebody has time to spend on it, it could be interesting 2009-01-23 16:56 convert from hg to git, however it would be lazy to do now 2009-01-23 16:57 the hg history will remain available, and the userspace development will be there 2009-01-23 16:57 yes 2009-01-23 16:58 the question is, will we start committing new changes to both git and hg? 2009-01-23 16:58 for kernel, I think probably we want the history in git 2009-01-23 16:58 I think we will use hg for now 2009-01-23 16:58 there must be an import 2009-01-23 16:59 yes 2009-01-23 16:59 without user/* 2009-01-23 16:59 we will need some patches like current 2.6.26.5 2009-01-23 16:59 like fs/Kconfig and fs/Makefile 2009-01-23 16:59 it is good to import the history, so people who did the work get credit 2009-01-23 17:00 yes 2009-01-23 17:00 we import all our mistakes too :) 2009-01-23 17:00 however, it would be lazy to filter out user/* (may be) 2009-01-23 17:01 so, I thought we may want to delay to import 2009-01-23 17:02 for now I will clone linus's tree and do my kernel builds with that instead of 2.6.26.5 2009-01-23 17:02 good 2009-01-23 17:02 however, fs/Kconfig and fs/Makefile change is needed 2009-01-23 17:03 yes of course 2009-01-23 17:03 so it will have a tux3 branch 2009-01-23 17:03 it's good for making patches 2009-01-23 17:03 ah, ok 2009-01-23 17:03 it's good 2009-01-23 17:04 so, we can make actual history to master 2009-01-23 17:05 ah 2009-01-23 17:05 I can do some trial history imports in that too 2009-01-23 17:05 ah, yes 2009-01-23 17:07 well, what I want to say, maybe I want to try to make history on some point 2009-01-23 17:07 and git tree for it 2009-01-23 17:07 good 2009-01-23 17:08 I think you will do a better job than me :) 2009-01-23 17:08 :) 2009-01-23 17:09 I'm not sure at all now, I have not been tried it before 2009-01-23 17:10 however, at least, I'm thinking it is possible 2009-01-23 17:11 I have heard git is good at that 2009-01-23 17:11 and also allows rewriting history 2009-01-23 17:12 oh, good 2009-01-23 17:12 which might be nice for cleaning up some old things, like using multiple different user names for the same person 2009-01-23 17:12 ah, good 2009-01-23 17:15 you need to sleep sometime ;) 2009-01-23 17:16 thanks :) oyasumi 2009-01-23 17:16 oyasumi 2009-01-23 17:16 and sorry about link change 2009-01-23 17:22 it's no problem whatsoever 2009-01-23 19:10 http://userweb.kernel.org/~hirofumi/add-fifo.patch 2009-01-23 19:10 how about this? it is untested yet though 2009-01-23 19:10 checking 2009-01-23 19:11 and name can be changed 2009-01-23 19:17 that is the style with explicit checks for empty and fully condition, and you have concealed the checks very expertly :) 2009-01-23 19:17 thanks :) 2009-01-23 19:20 I'll try to submit it after test after sleep 2009-01-23 19:23 struct defer { struct link *tail; u64 *pos, *top; }; <- I am working on a patch that does this 2009-01-23 19:24 could you merge after that one goes in? 2009-01-23 19:24 yes, of course 2009-01-23 19:24 your strategy is correct, and it avoids initializing in two places 2009-01-23 19:27 it is about init_defree? 2009-01-23 19:35 I was allocating and inserting a page in two places 2009-01-23 19:35 extra code :) 2009-01-23 19:37 and you have made it about as efficient as it can be: there is no sentinel element, and there is only a single pointer to the fifo 2009-01-23 19:39 yes, it is you did 2009-01-23 19:41 I just converted those to inline function 2009-01-23 19:43 -!- tim_dimm(~timothyhu@cpe-76-168-94-231.socal.res.rr.com) has joined #tux3 2009-01-23 19:44 earthquake 2009-01-23 19:44 btw, this algorithm is where com from? 2009-01-23 19:44 yup 2009-01-23 19:44 oh 2009-01-23 19:44 big one 2009-01-23 19:44 ish 2009-01-23 19:44 short and strong 2009-01-23 19:44 over 5.0 2009-01-23 19:44 but still not close to hirofumi's 2009-01-23 19:44 in osaka 2009-01-23 19:45 hirofumi, single linked circular lists are ancient, knuth talks about them 2009-01-23 19:45 well, Kobe was too big 2009-01-23 19:46 the linux innovation is to use container_of 2009-01-23 19:46 flips: damn felt pretty strong here 2009-01-23 19:46 yes, however, I didn't know fifo trick 2009-01-23 19:46 must have been close 2009-01-23 19:46 since it was so short 2009-01-23 19:47 shapor, no link yet? 2009-01-23 19:47 3.4 2009-01-23 19:47 1 mile from marina del ray 2009-01-23 19:47 wnw 2009-01-23 19:47 that's my house 2009-01-23 19:47 yeah 2009-01-23 19:47 seems like venice 2009-01-23 19:47 a little one, right under your toes 2009-01-23 19:47 http://earthquake.usgs.gov/eqcenter/recenteqsus/Maps/US2/33.35.-119.-117.php 2009-01-23 19:48 http://earthquake.usgs.gov/eqcenter/recenteqsus/Quakes/ci10373093.php 2009-01-23 19:48 http://quake.wr.usgs.gov/recenteqs/Quakes/ci10373093.html 2009-01-23 19:48 this time, the site didn't get knocked off the web like last time 2009-01-23 19:48 http://maps.google.com/maps?q=33.9841+-118.4701(M3.4+-+GREATER+LOS+ANGELES+AREA%2C+CALIFORNIA+-+2009+January+24++03%3A42%3A44+UTC)&ll=33.9841,-118.4701&spn=2,2&f=d&t=h&hl=e 2009-01-23 19:49 right by tim 2009-01-23 19:50 that's a block from switch studios 2009-01-23 19:50 wonder if matt was home 2009-01-23 19:53 opened a cupboard door here 2009-01-23 19:59 matt sez: whoomp 2009-01-23 19:59 straight up and down 2009-01-23 20:04 twice 2009-01-23 20:04 whoomp de whoomp 2009-01-23 20:04 shake and bake 2009-01-23 20:43 -!- tim_dimm(~timothyhu@cpe-76-168-94-231.socal.res.rr.com) has joined #tux3 2009-01-23 21:13 tim_dimm: how was it by you? 2009-01-23 21:13 it was the same here 2009-01-23 21:14 less than 3 secs probably 2009-01-23 21:14 up and down twice