16:04:57) Alster: yo
(16:05:34) Alster: txopi, didleth, didleth1, harcesz, dez`afk : meeting time
(16:05:58) didleth1: yes, hello everyone
(16:06:10) txopi: yap
(16:06:12) Alster: hello did
(16:06:21) Alster: hello txopi
(16:06:25) txopi: Alster, what do you know about hyperactive?
(16:06:42) Alster: i know it's not on the agenda. :)
(16:06:46) txopi: at eh we are planning to migrate to drupal
(16:06:56) Alster: i know little about it really
(16:07:11) didleth1: txopi: as i know hyperactive is cool, but there is no so much people who are working on it like in drupal
(16:07:39) txopi: hyperactive is mantained by yossarian and seems to be easy to install a site, but it is mantained but just one person as far as i know (that isn't risky?)
(16:07:55) didleth1: if we are discusing about migration in imc-pl, the argument for dfrupal was: there is a lot of peopole who know this, and hyperactive knows only a few people
(16:08:33) Alster: that's my main concern, too. there are actually other developers, too, but i think it will go amess if yoss should ever meet the bus factor
(16:08:37) txopi: drupal is really powerful, so it is also more complicated (permissions, modules problems...)
(16:08:46) txopi: all options has its pros and cons
(16:08:56) harcesz: huray hurah itz a meating! :D
(16:09:06) txopi: bus factor?
(16:09:26) harcesz: hi alll
(16:09:28) harcesz: *all
(16:09:39) didleth1: txopi: but there is more poeple who knows drupal i belive
(16:09:48) Alster: txopi: that's a little tech 'joke'. when someone gets hit by a bus, they can no longer work, and that'S the 'bus factor'
(16:09:56) didleth1: so its more easy to find help now, in the beging, and secureate this after
(16:10:00) txopi: xDD
(16:10:04) txopi: i understand
(16:10:04) txopi: ok
(16:10:11) txopi: we can start the meeting
(16:10:20) Alster: \o/
(16:10:25) didleth1: i'm afraid it cajn be difficult to securate hyperactive when people who develop this will not working on this any more
(16:10:27) txopi: do we have any agenda?
(16:10:32) didleth1: that is myu opinion
(16:10:50) Alster: hello harcesz, too
(16:10:58) Alster: the agenda is on the topic
(16:10:59) txopi: previous session logs: https://we.riseup.net/kosmos/skill-sharing-session-9-log
(16:11:06) harcesz: hi Alster
(16:11:20) Alster: btw didleth, never mind your internet connection...
(16:11:21) Alster: <r7> our internet back after 2 days. the transmitter on the mountain opposite blew over :-(
(16:11:33) harcesz: ouch
(16:11:48) txopi: this is 10th session
(16:11:57) didleth1: you mean jordie is not avaliable any more?
(16:11:58) txopi: we don't have any writen agenda as far as i know
(16:12:04) Alster: oh you're right harcesz
(16:12:16) didleth1: :-/ i hope i wasn't who had to doing agenda?:D
(16:12:20) Alster: didleth1: no this is not related to jordie
(16:12:46) txopi: we can use jordie just 11 more days
(16:12:53) Alster: hmm it may have been me who volunteered to do the agenda
(16:13:11) txopi: jan 07 22:03:49 <Alster> oh, something else i need to announce regarding backups: i may have much limited bandwidth starting january 21. so may no longer be able to provide backup space via jordie. :-/
(16:13:12) didleth1: our internet back after 2 days. the transmitter on the mountain opposite blew over :-( so.... can you say it in germna?
(16:13:15) didleth1: *german
(16:13:29) Alster: txopi: well maybe you can use it longer than for 11 more days, but i cannot guarantee it at this point
(16:13:38) txopi: aha
(16:13:49) Alster: didleth1: i'll explain in private message
(16:14:09) didleth1: ok
(16:14:12) harcesz: Alster: we got a new VPS available
(16:14:30) harcesz: we could stress test it with the backups if needed
(16:15:35) Alster: congrats :)
(16:15:53) harcesz: (it's hosted by bzzz.net which is a polish hosting collective and it would be better for imc-pl not to have a polish admin, but for backups I dont think there would be a problem)
(16:16:17) Alster: sounds good.
(16:16:22) didleth1: harcesz: bzzz.net ? its safe?
(16:17:06) harcesz: as safe as any other collective
(16:17:10) didleth1: you said every server is so safe like his weakest part
(16:17:41) txopi: what it means VPS?
(16:18:12) harcesz: -> prv (some internal polish issues)
(16:18:15) txopi: you say that backups could be made on bzzz.net?
(16:18:21) harcesz: txopi: virtual private server
(16:18:29) harcesz: something by the line of xen
(16:18:35) didleth1: :-/
(16:18:39) txopi: harcesz: ok
(16:18:40) harcesz: (it might actually be running on xen :P)
(16:18:58) didleth1: i'm not sure - in bzzz.net there are some sites who are not good secured as i know - wouldnt it be a problem?
(16:19:11) Alster: VPS=virtual (private) server
(16:19:29) Alster: oh harcesz already explained it sorry
(16:19:33) Alster: so what do we do about the missing agenda and logs?
(16:20:07) txopi: logs are not missing: https://we.riseup.net/kosmos/skill-sharing-session-9-log
(16:20:14) didleth1: aha, ok, harcesz explainted me - it seems the site I wroted about is in another server
(16:20:20) didleth1: so it should be ok
(16:20:33) txopi: aha
(16:21:37) didleth1: ok, so lets go to agenda
(16:22:12) txopi: we can define the agenda here
(16:22:29) ***harcesz is still waiting to confirm it with the person responsible for the server, but I wouldn't believe he'd have any problems with that
(16:23:34) txopi: 1) Set time constraints for this meeting
(16:23:37) txopi: 2) Do we want a summary or IRC logs of this meeting? If so, who volunteers to do it?
(16:23:37) txopi: 3) Next meeting, and who volunteers to prepare the agenda for it?
(16:23:37) txopi: 4) Server hosting
(16:23:37) txopi: 5) Skill Sharing: backupninja, etc.
(16:23:57) txopi: harcesz, that's great. go on with that issue please
(16:24:32) txopi: i am also trying to store backups at mundurat.net (but i have no response yet)
(16:24:56) txopi: do you agree with my agenda proposal?
(16:25:05) #kosmos: tryb (+oo didleth didleth1) przez txopi
(16:25:12) Alster: very nice, thanks txopi
(16:26:33) didleth1: for me ok txopi
(16:26:34) harcesz: txopi: willdo
(16:26:49) ***txopi is creating the wiki page on crabgrass
(16:27:00) Alster: regarding 1: i need to leave at 18:00 but could return ~19:00 for some more hours
(16:27:01) txopi: so we can continue
(16:27:40) didleth1: ok, so:
(16:27:47) didleth1: 1) Set time constraints for this meeting
(16:27:57) didleth1: so first part till 18.00
(16:28:12) didleth1: the second part - 19.00 till....how is good tou you?
(16:28:43) Alster: 'you' being whom?
(16:28:52) Alster: all of us?
(16:28:57) didleth1: yes, all of you
(16:29:00) txopi: https://we.riseup.net/kosmos/skill-sharing-session-10-agenda
(16:29:28) ***Alster hugs txopi
(16:29:34) didleth1: thatwhy english is a bed language.... it even has no difference betwqeen plural and saingular
(16:29:54) txopi: i would like not to spend all the afternoon in the irc, but agrees
(16:30:25) didleth1: so maybe 16-18; than 19.00-20.00 or 19.00-21.00?
(16:30:36) didleth1: which is better for you?
(16:30:40) harcesz: didleth you can use the EN-IE "yous"
(16:30:54) harcesz: txopi: +1
(16:31:06) didleth1: harcesz: this mean 'wy' in polish/ihr in german?
(16:31:53) harcesz: maybe lest focus on starting rather than discussing the length of it? if we make it faster then it's good, if not, we will decide then
(16:32:20) harcesz: -> prv
(16:32:29) Alster: i think that's a good suggestion
(16:32:34) didleth1: ok
(16:33:12) Alster: (and just because i can't be there between 6 and 7 doesn't mean you have to stop, too.)
(16:33:27) Alster: 2) Do we want a summary or IRC logs of this meeting? If so, who volunteers to do it?
(16:34:00) didleth1: maybe harcesz? ;]
(16:34:25) Alster: alternatively, i can do the logs today
(16:34:32) txopi_: hi again
(16:34:39) ***didleth1 is malignat ;]
(16:34:40) txopi_: i forgot to plug my desktop :-/
(16:34:44) txopi_: i'm back
(16:34:45) Alster: txopi++
(16:34:48) użytkownik txopi wyszedł z pokoju (quit: Read error: No route to host).
(16:34:53) #kosmos: tryb (+o txopi_) przez didleth1
(16:34:55) txopi_ nazywa się teraz txopi
(16:35:12) harcesz: didleth if u wouldn't mind, i'd appreciate if u could do it
(16:35:22) txopi: for me 16-18 + 19-20 is ok
(16:35:40) didleth1: ok so i do logs
(16:35:40) harcesz: so, lets get on with it! http://www.youtube.com/watch?v=dEtm_Q2LK9g !
(16:36:38) Alster: thanks
(16:36:38) txopi: ok
(16:36:43) txopi: 3) Next meeting, and who volunteers to prepare the agenda for it?
(16:37:18) Alster: so we need both a meeting planner and agenda
(16:37:29) didleth1: i can do meeting planer
(16:37:29) didleth1: but i prefer agenda do someone another
(16:38:00) Alster: i think someone volunteered for the meeting planner last time already
(16:38:01) txopi: we can decide the agenda in the end of this session
(16:38:05) Alster: but i can be wrong there
(16:38:16) txopi: i don't remember
(16:38:20) didleth1: when, axaclty... i have a nerw idea
(16:38:23) txopi: are you talking about me?
(16:38:38) didleth1: as i see last time agenda is this same
(16:39:03) Alster: i'll do the agenda for next time if that's ok with you.
(16:39:18) didleth1: so maybe just not change aganda and no create the new one, only - when some decide there is a new topic to talk about - just put it into agenda?
(16:40:43) Alster: yes, i'll just update it slightly if needed. or add some new topics (or those we haven't finished yet) as needed.
(16:40:53) txopi: i think it is good to know the agenda before the meeting
(16:41:37) txopi: sometimes i try to bring people from eh showing them the agenda :-)
(16:42:00) Alster: btw i made meeting 9's log public, hope that'S ok with you
(16:42:24) Alster: ok, next topic then?
(16:42:33) txopi: didleth1, so you will update the planner?
(16:42:49) txopi: thanks Alster
(16:43:25) didleth1: txopi: yes o prepere the planer for the next week - i only not sure or i will have time on the next week
(16:43:38) didleth1: ok, next topic
(16:43:59) didleth1: 4) Server hosting
(16:44:20) txopi: i have some questions here
(16:44:52) txopi: production server(s): kosmos and in the future nadir's VPS
(16:45:44) Alster: didleth1: please also notify us when the planner is ready for voting
(16:45:47) txopi: backup server(s): jordie (not finished and will end soon) and possibly also bzzz's VPS and mundurat's acount
(16:45:52) txopi: also "salt"?
(16:46:01) didleth1: ok, i will sent a message to the list
(16:46:35) Alster: thanks didleth
(16:46:35) txopi: if someone has news about tachanka, please talk
(16:47:24) txopi: please correct me if i am wrong in any detail or explain a bit about salt
(16:49:05) Alster: salt is a Xen based virtual server I host. It is shared with other Indymedia projects. I'm the only one having root there.
(16:49:28) Alster: it is used for backups and as a shell server.
(16:49:40) txopi: ahha
(16:49:47) Alster: also for subversion via ssh and http/webdav
(16:50:59) txopi: i think it would be god if we achieve to make backups at bzzz and mundurat (two backup places)
(16:51:06) Alster: it is well connected but located on a rented server in a commercial colocation facility. the entire virtual server is stored on an encrypted partition. i'm offering it as a secondary backup location.
(16:51:25) txopi: but if we can't or we don't achieve it soon, we can also use salt
(16:51:48) didleth1: harcesz: when server you are talking about would be ready?
(16:51:51) txopi: this way we will use all indymedia's resources just if needed (so other people can use them)
(16:51:59) didleth1: i mean - when can we start backup on this vps?
(16:52:08) harcesz: didleth1: no idea, havent confirmed it yet
(16:52:44) didleth1: so i belice its good to thing about salt
(16:52:56) harcesz: also I wouldent expect it to be encrypted atm
(16:53:08) txopi: belice?
(16:53:19) didleth1: sorry, i belive it should be
(16:54:09) txopi: ah, ok
(16:54:36) Alster: salt:~# LANG=C du -sch /home/imckosmos
(16:54:36) Alster: 14G /home/imckosmos
(16:54:36) Alster: 14G total
(16:54:36) Alster: salt:~# LANG=C df -h /home/imckosmos
(16:54:36) Alster: Filesystem Size Used Avail Use% Mounted on
(16:54:37) Alster: /dev/sda6 50G 42G 4.8G 90% /home
(16:54:41) Alster: salt:~#
(16:55:05) Alster: salt has limited space available, but there are already old kosmos backups stored there which can be removed
(16:55:17) txopi: mmmh
(16:55:34) didleth1: yhm
(16:55:34) txopi: but 50G is enought for backups?
(16:55:47) txopi: as i understood it sin't (but i can be wrong)
(16:55:49) didleth1: i belive it should be enoguht
(16:56:02) txopi: as i understood it isn't (but i can be wrong)
(16:56:26) didleth1: i mean, if we asked nadir about 60 gb or about 100 gb that we could have a backups, it means even 40 gb would be enoguht for it
(16:56:33) harcesz: txopi: theres 4.8 available there at the moment
(16:56:49) didleth1: harcesz: but we can remove the old backups
(16:57:09) harcesz: yeah old kosmos backups and thats not the only thing thats there
(16:57:27) txopi: at rakentz's machine we had 35G and we decided it wasn't enough
(16:58:54) Alster: it turns out if we used rdiff-backup and also create the the database dump backups in a way that allows for incremental backups 20G _may_ be enough
(16:59:44) Alster: but if we later decide to switch to some other backup mechanism which doesn'T allow for remote incremental backups it won't suffice.
(17:00:18) Alster: i can actually increase the available size on salt if it gets used
(17:00:20) txopi: you mean not using backupninja?
(17:01:10) Alster: we could still use backupninja but use another handler such as duplicity if the production servers' internet connection allows for it and we decide we prefer encrypted remote backups.
(17:01:21) ***harcesz strikes out the last point of todays agenda :P
(17:01:34) txopi: ok
(17:01:37) harcesz: ah, so /me unstrikes :P
(17:01:57) txopi: so we should work putting salt ready
(17:02:00) txopi: use it
(17:02:17) txopi: with duplicity and whatever you think we should do
(17:02:40) txopi: and then, look how many space we need and search for other backup locations at bzzz and mundurat
(17:02:45) txopi: do you agree?
(17:02:46) Alster: i can get you more space on it tonight
(17:02:57) Alster: sounds good to me
(17:03:19) didleth1: for me ok
(17:03:43) harcesz: nice one Alster, thx
(17:03:46) Nazywasz się teraz Didleth
(17:04:09) txopi: ok, more things about backup servers?
(17:05:01) Alster: salt is not as much of an issue as it may seem, it may be offline for some days and return with limited bandwidth at first, but it will be back to full speed 'soon'
(17:05:21) txopi: ...
(17:05:48) txopi: well, we will do what we can...
(17:06:15) txopi: about production servers, today i will send a message to nadir asking for the vps
(17:06:27) txopi: i will send a copy to you if you want
(17:06:42) txopi: harcesz, do you use gnupg for mail encryption?
(17:06:49) harcesz: txopi: yeah
(17:07:04) harcesz: oh, btw Alster youre using an out-of-date signature
(17:07:05) txopi: please, send me your public key please
(17:07:21) harcesz: at least according to my pgp management
(17:07:54) Alster: harcesz: please update your copy of my publix key from hkp://zimmermann.mayfirst.org
(17:08:10) harcesz: k
(17:08:23) Alster: harcesz: please update your copy of my publix key from hkp://zimmermann.mayfirst.net
(17:08:35) Alster: uh sorry ignore the last line
(17:10:01) Alster: so we're done with the server topic?
(17:10:07) Alster: next will be 5) Skill Sharing: backupninja, etc.
(17:10:12) txopi: ok
(17:10:15) Didleth: ok
(17:11:19) Alster: i think there's two options which would make sense for now
(17:12:03) Alster: (a) look into how we can efficiently add backups of the database to jordie, and optionally apply this to salt if it works out
(17:12:53) Alster: (b) setup a backup to salt just like we have it for jordie right now, and topionally also look into how to efficiently create database dumps to salt and jordie
(17:13:12) Alster: so basically the difference is just what the focus is on
(17:13:26) Alster: any other suggestions?
(17:13:30) Didleth: i'm not sure, but
(17:13:35) Didleth: kosmos is badly conected
(17:13:58) Didleth: so wasn't is easier to backup the rest to the jordie and than from jordie to salt?
(17:14:07) Didleth: brb/go to badroom
(17:14:45) Alster: i assume the bad room is a bathroom
(17:14:48) harcesz: Didleth: if this backup fails the other wont be of any use either
(17:15:23) harcesz: Alster: yeah, sound transcription
(17:16:03) txopi: if we choose b, we will change a bit kosmos config and then wait a lot of hours until salt is syncronized
(17:16:18) Alster: what harcesz just explained is why it doesn't offer a remarkably increased reliability if we do it like this so I'd recommend not to do it this way.
(17:16:56) txopi: if we choose a we can learn how to do database backups but we will copy a lot of files for nothing (because in some days we will copy that information to other place)
(17:17:18) Alster: also, the rdiff-backup backups we are doing now are _very_ bandwidth efficient. remember that the incremental backups (no DB dumps, though) were just ~5 MB
(17:17:40) txopi: ok, so option a?
(17:18:25) Alster: personally i think (a) is the better choice
(17:18:34) harcesz: +1 for a
(17:19:13) Alster: ok, so let's do it this way
(17:19:22) txopi: a is ok for me
(17:19:35) Alster: let's share a screen on kosmos but report back here what we do
(17:20:19) Alster: a root owned screen on kosmos is setup, please join
(17:20:33) Alster: ...and report back here when you did
(17:21:36) Alster: Didleth: you don't have to join, i'll run script again (but it's up to you of course)
(17:22:08) txopi: i am ready
(17:23:25) Alster: # script -a -f skill-sharing-session_2010-01-10_kosmos.script
(17:23:25) Alster: Script started, file is skill-sharing-session_2010-01-10_kosmos.script
(17:23:32) Alster: harcesz, Didleth ?
(17:24:21) harcesz: Alster: one minute
(17:25:43) Didleth: blah, badenzimmer ;P
(17:25:49) Didleth: fucking english ;>
(17:27:13) Alster: harcesz: if we can help with anything please let us know
(17:27:17) harcesz: just to check before a dissaster comes, should I just normally start screen or what?
(17:27:28) Alster: sudo screen -x
(17:27:43) harcesz: Alster: I had to check my key password which takes a few secs ;]
(17:27:47) Didleth: i will join
(17:27:52) Alster: i see :)
(17:28:08) Alster: i guess we cannot help there ;-)
(17:29:28) harcesz: k, i'm in
(17:29:41) Alster: ok, apprently someone is typing there
(17:29:51) harcesz: :P ups.
(17:30:04) ***harcesz moves away from the keyboard
(17:30:06) Didleth: :P blah, to much windows
(17:30:13) Alster: hehe
(17:30:24) Didleth: i will try to join from odysesz
(17:30:28) Didleth: and vergleichen
(17:30:58) Alster: can we decide who drives this screen?
(17:31:19) Alster: i don't need to but it would be good if it was a single person
(17:31:32) Didleth: btw how can i move up to see what wahs wroten, if pgup doesnt work?
(17:31:53) Didleth: i can if someone will tell me what to write there
(17:32:12) Alster: Didleth: just run 'less' on the script log in another window
(17:33:11) Alster: screen also allows for inspecting the backup some way but it would need to be done for all of us
(17:33:12) Didleth: in another windows, so you meaned not in thois where we are?
(17:33:22) Alster: Didleth: exactly
(17:33:42) Didleth: but in screen, right?
(17:33:51) Alster: up to you.
(17:34:14) Alster: either a new screen window or a seperate terminal window + ssh connection from you to kosmos
(17:34:39) Didleth: ok, so i wll just open it in another window
(17:34:48) Alster: (there is also a way to share existing ssh connections in a different terminal window but i forgot the ssh usage for that)
(17:34:54) Alster: ok
(17:35:27) Alster: so let's start with having a look at the backupninja website and man page
(17:36:07) txopi: ok
(17:36:10) Alster: https://labs.riseup.net/code/projects/show/backupninja/
(17:36:26) Didleth: in which director is the script?
(17:36:26) Alster: that's the website, please have a look
(17:36:33) Didleth: i cant find it
(17:36:46) Alster: Didleth: /root/skill-sharing-session_2010-01-10_kosmos.script
(17:38:05) Alster: everyone done reading the web page?
(17:38:11) Alster: any questions regarding it?
(17:38:25) txopi: yes
(17:38:30) Alster: i will talk about ninjahelper in a minute
(17:38:49) Didleth: yhm
(17:38:54) Alster: so if it's anything else please ask
(17:39:19) txopi: so rdiff-backup and duplicity are other utilities not related with backupninja
(17:39:30) txopi: but backupninja use them, is it right?
(17:40:03) Alster: exactly
(17:40:28) Alster: backupninja is a management frontend/wrapper for the 'real' backup utilities
(17:40:51) txopi: if you don't know that utilities but know backupninja and how to configure it, the rest is transparent for you
(17:40:56) txopi: ok
(17:41:01) Alster: rdiff-backup and duplicity are such backup utilities, but it could also be ftp-backup or whatever
(17:41:09) txopi: aha
(17:41:12) Alster: (or just plain 'ftp')
(17:41:19) txopi: ok, i understand
(17:41:28) Didleth: i soppose to understand it too
(17:42:26) Alster: ok, we already had alook at these config file snippets backupninja (I will call it BN in the future) uses for its handlers
(17:43:09) Alster: i explained that you can copy the example handlers from /usr/share/docs/backupninja/examples to /etc/backup.d/ as needed
(17:43:27) txopi: aha
(17:43:37) Alster: /usr/share/doc/backupninja/examples/ is the correct path
(17:43:49) Alster: # ls /usr/share/doc/backupninja/examples/
(17:43:49) Alster: backupninja.conf example.makecd example.rdiff example.sys
(17:43:49) Alster: example.dup example.mysql example.sh example.trac
(17:43:49) Alster: example.ldap example.pgsql example.svn
(17:43:49) Alster: kosmos:~#
(17:44:35) Alster: a more easy way to setup and manage backupninja handlers can be the ninjahelper utility which comes with BN
(17:45:25) Alster: it provides a text graphics based user interface based on the ncurses library
(17:45:35) txopi: aha
(17:45:58) Alster: (most text graphics user interfaces are based on this, such as aptitude when run in the interactive mode, too)
(17:45:58) txopi: there is a screensplash here: https://labs.riseup.net/code/wiki/backupninja/Usage
(17:46:14) Alster: great :)
(17:46:24) Alster: i'm running ninjahelper in the screen now
(17:46:39) Alster: just issue 'ninjahelper' to start it
(17:47:15) Alster: the main menu displays the existing handler configurations, basically the contents of /etc/backup.d
(17:47:38) harcesz: Alster: hm, ony thing I can see is "cat 10_syste..."
(17:47:43) Alster: it allows for adding new handlers and for editing/deleting/testing/running existing handlers
(17:47:45) harcesz: wrong screen? :]
(17:48:11) Alster: yes, wrong screen harcesz
(17:48:16) txopi: i see the blue and gray screen
(17:48:17) Didleth: i see
(17:48:25) Didleth: its corect?
(17:48:32) harcesz: k, got there
(17:48:34) Didleth: /root/skill-sharing-session_2010-01-10_kosmos.script
(17:48:34) Didleth: kosmos:~#
(17:48:34) Alster: harcesz: but you didnt miss much, i just invoked 'ninjahelper' so far
(17:48:40) Alster: Didleth: correct
(17:49:26) Alster: harcesz: see the note i left in your screen
(17:49:52) harcesz: Alster: yeah, got it, I'm on the right one
(17:49:55) harcesz: thx
(17:50:16) Alster: ok, i'm going to pretend to setup a new rdiff handler, just to show you the interface
(17:50:36) Alster: see the options we have there
(17:50:46) txopi: aha
(17:50:46) Alster: │ │ dup incremental encrypted remote filesystem backup │ │
(17:50:47) Alster: │ │ mysql mysql database backup │ │
(17:50:47) Alster: │ │ pgsql postgresql database backup │ │
(17:50:47) Alster: │ │ ldap ldap database backup │ │
(17:50:47) Alster: │ │ makecd makecd backup │ │
(17:50:47) Alster: │ │ rdiff incremental remote filesystem backup │ │
(17:50:49) Alster: │ │ sys general hardware and system info │ │
(17:51:15) Didleth: o dont see it :-/
(17:51:30) Alster: i just pasted it above Didleth
(17:52:01) Didleth: yhm, no i see on screen
(17:52:02) Alster: now in the submenu for creating a new rdiff-backup handler we have this:
(17:52:04) Didleth: lag again
(17:52:08) Alster: ┌──────────────────────rdiff-backup action wizard─────────────────────────┐
(17:52:08) Alster: │ choose a step: │
(17:52:08) Alster: │ ┌─────────────────────────────────────────────────────────────────────┐ │
(17:52:08) Alster: │ │ src choose files to include & exclude │ │
(17:52:09) Alster: │ │ dest configure backup destination │ │
(17:52:13) Alster: │ │ conn set up ssh keys and test remote connection │ │
(17:52:15) Alster: │ │ finish finish and create config file │ │
(17:52:45) Alster: 'conn' automates the process of setting up a passwordless ssh connection to a remote host for you
(17:52:54) Alster: this is quite handy, saves you some typing
(17:53:03) txopi: aha
(17:53:27) Alster: it doesn'T work if the remote host port differes from the default of 22, though
(17:53:40) Alster: i'm not doing it now since we already have this setup to jordie and salt
(17:54:01) txopi: aksi sakt?
(17:54:11) txopi: also salt?
(17:54:16) Alster: yes
(17:54:29) txopi: they where two handlers, sys and jordie
(17:54:49) txopi: jordie has two destinations?
(17:54:53) Alster: oh we don'T have a backup going to salt currently, but we have passwordless ssh access configured to salt
(17:55:04) txopi: aaah
(17:55:05) txopi: ok
(17:55:44) Alster: the source definition looks like the corresponding section in the rdiff-backup handler configuration file we looked at last time
(17:56:09) Alster: you can set several 'include' statements
(17:56:21) Alster: and several 'exclude' statements
(17:56:32) txopi: aha
(17:56:44) Alster: now that i have setup this part it is marked as (DONE)
(17:57:04) Alster: i can still edit /change it by reseleting the 'src' option
(17:57:21) txopi: aha
(17:57:27) txopi: easy
(17:57:39) Alster: but i cannot finish setting up this new rdiff-backup handler yet since it is not completely configured, yet
(17:57:48) Alster: we need yet to configure the destination
(17:58:19) Alster: 'keep' defines how long to keep backup history on the destination/backup server
(17:58:26) Alster: 60D is 60 days
(17:58:36) txopi: ok
(17:58:37) Alster: you could also use 1Y
(17:58:48) Alster: but that would be a bit long ;-)
(17:59:10) txopi: dest_type = remote?
(17:59:39) Alster: last session we discussed that with rdiff-backup, you can do either push or pull style backups
(17:59:56) txopi: ah yes, i remember
(18:00:07) Alster: if the destination is a remote destination then you push your backups
(18:00:27) Alster: if it is local then you pull the backups in from a remote source
(18:00:35) txopi: ok
(18:00:53) Alster: oops, i found a bug
(18:01:01) txopi: yes i saw!
(18:01:19) harcesz: missed it
(18:01:23) Alster: this backupninja version is a bit old, it's in debian stable after all.
(18:01:26) harcesz: pics or it ddnt happen
(18:01:43) harcesz: oh, thats only like 2 years old or so ;]
(18:02:24) Alster: when i hit return to confirm the (yet incomplete) settings, the 'remote' string was written to the 'host' field
(18:03:03) Alster: we can consider to use a more recent backupninja version, the one from unstable normally works on stable, too
(18:03:55) Alster: so i was again trying to complete setting up my new handler without having setup everything. the ssh connection is actually missing.
(18:04:06) Alster: at least ninjahelper thinks so
(18:04:39) Alster: and in safeguarding me to have a complete backup configuration, it is actually annoyning me now since i already know i have the ssh connection setup properly
(18:05:05) Alster: so whether or not you use ninjahelper depends on your personal preferences
(18:05:29) Alster: it makes it easy at first, but i'd recommend you learn to work with the configuration files directly, too
(18:06:06) Alster: i will 'cancel' this configuration now since we don't need it, it was just for testing.
(18:06:10) txopi: perhaps in the new version this step isn't obligatory
(18:06:24) Didleth: (the only thing i see in screen is rdiff-backup action wizard btw)
(18:06:32) Alster: that's possible. it sure has many fixes + improvements
(18:07:07) Alster: ...and i'm leaving for now , see you here at 19.00
(18:07:16) txopi: ok Alster
(18:07:23) harcesz: cya
(18:07:24) txopi: thank you for the tour! :-P
(18:07:26) harcesz: have fun
(18:07:47) Didleth: till 19.00
(18:07:49) txopi: i'm gonna do some things too, if you don't mind :-)
(18:08:04) Didleth: i will prepare some dinner
(18:09:27) Alster: feel free to play around a bit with backupninja and create some new configs for fun. just try nto to touch the existing two configs, please.
(19:02:22) Didleth: Alster: and can i clik 'finish and create config file'?
(19:02:33) Didleth: i belive my window is stopnig
(19:04:37) Alster: Didleth: yes you can :)
(19:05:01) Alster: i'll be with you in a minute
(19:05:24) Didleth: blah, i was remote
(19:05:44) Alster: (me is still teaching how to bring windows to the front to mommy)
(19:06:58) txopi: :-D
(19:08:36) txopi: Didleth, i don't find your gnupg key. can you send me?
(19:09:23) Alster: ok, i'm back, sorry for the delay
(19:09:32) Didleth: txopi: it should be unter 'didleth'
(19:09:55) Didleth: there is only the one didleth min internet, so im you ifnd a gpg key in some server, tis mine
(19:10:29) txopi: ok
(19:10:54) txopi: Didleth (Rety ile zamieszania z tymi kluczami. I tak im imienie i nazwiska nie podam.) <didleth@riseup.net>
(19:11:04) Didleth: yes, thisone :]
(19:11:13) Didleth: i hope its still important
(19:12:20) txopi: Alster, Didleth, harcesz: i have send the e-mail to Nadir asking for the domU
(19:12:32) txopi: you will receive a copy
(19:12:42) Didleth: ok, thx txopi
(19:13:04) Alster: thanks
(19:13:21) Alster: ok, so we continue with backupninja?
(19:13:48) txopi: yes please
(19:14:04) Alster: harcesz: in case you want to take part, we're continuing
(19:14:04) Didleth: ok i give new action, than pgsql database backup, thatn directory where to store the backups - can i just give /var/backups/postgres/test ?
(19:14:12) Didleth: or do i have to select some existing?
(19:14:31) Alster: Didleth: i don't really know. I _assume_ it must be existing
(19:15:02) Didleth: and if i select default, also │/var/backups/postgres would i broke something?
(19:15:24) harcesz: i'll be back in few minutes, you can start without me, i'll catchup
(19:15:33) Alster: ok :)
(19:15:47) Alster: Didleth: you could check what's in this directory right now
(19:16:00) Alster: to make sure you don't overwrite something accidentially
(19:17:16) Didleth: didleth@kosmos:~$ sudo ls /var/backups/postgres
(19:17:16) Didleth: kosmos-all.sql.gz
(19:17:38) Didleth: maybe its more safe in my case just create some test-file and try with this
(19:17:44) Alster: is this a recent backup?
(19:18:02) Alster: if not, you could just delete it and safely use this directory
(19:18:14) Didleth: Alster: i'm not sure
(19:18:23) Didleth: but i will see how many space it ave
(19:18:28) Alster: run 'ls -l' on it
(19:18:29) Didleth: *has
(19:18:48) Alster: this should show both the last modification time and the size
(19:19:02) Didleth: -rw------- 1 postgres postgres 5283873713 Dec 19 06:01 kosmos-all.sql.gz
(19:19:04) Alster: 'ls -lh' would show the size in a more readable format
(19:19:13) Didleth: i belive its a ood idea to don't do nothing with this
(19:19:26) Alster: i think it's safe to delete this
(19:19:31) Didleth: i used to do ls - falh ; )
(19:19:33) Alster: i am very sure it is
(19:19:45) Didleth: ok, so i can use it to test. right?
(19:19:49) Alster: yes
(19:20:13) Didleth: ok i come to mre-human-language to be sure
(19:20:42) Didleth: kann ich das benutzen ohne etwas geafehrliches zu tun?
(19:21:00) Alster: it is safe to continue with the settings you have
(19:21:09) Didleth: ok
(19:21:42) Didleth: (if i broke something, only 3 people will kill me...erst than another 3000 .. ;P)
(19:22:11) Alster: they will have to kill me first
(19:22:17) Didleth: blah/..something freez again
(19:22:20) Alster: and besides, there are backups elsewhere
(19:22:21) txopi: 2 is you one
(19:22:47) Alster: ┌───────────────────────────────main menu─────────────────────────────────┐
(19:22:48) Alster: │ Select a backup action for more options, or create a new action: │
(19:22:48) Alster: │ ┌─────────────────────────────────────────────────────────────────────┐ │
(19:22:48) Alster: │ │ 1 /etc/backup.d/10_systeminfo.sys │ │
(19:22:48) Alster: │ │ 2 /etc/backup.d/20.pgsql │ │
(19:22:50) Alster: │ │ 3 /etc/backup.d/20_eh_indy.pgsql │ │
(19:22:52) Alster: │ │ 4 /etc/backup.d/90_jordie.rdiff │ │
(19:22:54) Alster: │ │ new create a new backup action │ │
(19:22:56) Alster: │ │ quit leave ninjahelper │ │
(19:23:00) Alster: there are now two .pgsl handler configurations
(19:23:17) Didleth: ok stupid question (belive me, if harc is around there reale are stupid question) - what i should do to put ok?
(19:23:18) Alster: so two backups will be created supposedly
(19:23:38) Didleth: enter, --> and k doesnt work
(19:23:51) Alster: enter should work
(19:23:57) Didleth: ok now it did, lag again
(19:23:59) txopi: Didleth, wait!
(19:24:00) Alster: unless you're on cancel
(19:24:03) Alster: which you are now
(19:24:11) txopi: you pushed -> key
(19:24:12) Alster: go left
(19:24:17) Alster: with the arrow keys
(19:24:19) txopi: so the focus is over Cancel
(19:24:35) Alster: thanks txopi, i missed it
(19:24:39) txopi: so you should push <- key to return focus to OK
(19:24:43) txopi: ok
(19:24:50) txopi: now push Enter
(19:24:59) txopi: in your keyboard
(19:25:01) Didleth: sorry txopi, it just jumped to cancel when i put arrow again and again and again
(19:25:09) Alster: well done Didleth
(19:25:27) Didleth: 1 /etc/backup.d/10_systeminfo.sys │ │
(19:25:27) Didleth: │ │ 2 /etc/backup.d/20.pgsql │ │
(19:25:27) Didleth: │ │ 3 /etc/backup.d/20_eh_indy.pgsql │ │
(19:25:27) Didleth: │ │ 4 /etc/backup.d/90_jordie.rdiff
(19:25:28) Alster: i wonder what 'Cancel' would do here anyways, it doesn't make much sense on the main menu
(19:25:31) Didleth: thit is what i see
(19:25:36) Didleth: should i give ok?
(19:26:00) txopi: Alster, perhaps same as quit
(19:26:04) txopi: or not...
(19:26:11) Didleth: well
(19:26:13) Alster: txopi: right :)
(19:26:17) Didleth: i can give cancel now?:]
(19:26:23) Didleth: and test what it works?:]
(19:27:00) Alster: i say run 'OK' on 'quit' and then restart ninjahelper
(19:27:06) Didleth: ok
(19:27:11) Alster: and try 'Cancel' then
(19:27:36) Alster: you just pressed 'OK' and thus entered the currently selected action configuration
(19:27:42) Didleth: o, there even is a option
(19:27:46) Didleth: 'do a test run'
(19:28:05) Alster: you are currently configuring the 10_systeminfo.sys handler
(19:28:12) Didleth: :-/
(19:28:21) Alster: return to the main menu by pressing enter
(19:28:21) Didleth: you said i should put 'ok'
(19:28:26) Didleth: ok
(19:28:29) Didleth: sorry
(19:28:34) Alster: <Alster> i say run 'OK' on 'quit' and then restart ninjahelper
(19:28:49) Alster: i said 'OK' on 'quit'
(19:28:55) Alster: so move down to 'quit'
(19:28:57) Alster: then OK
(19:29:10) Didleth: ok - did i broken up something?
(19:29:16) Alster: not at all
(19:29:26) Alster: ok now restart ninjahelper
(19:29:49) Alster: up cursor twice
(19:30:02) Alster: and enter :)
(19:30:28) Alster: it's no longer in the history now
(19:30:29) txopi: down cursos until you find ninjahelper
(19:30:44) Alster: well could be
(19:30:48) txopi: or just write ninja and press tab key twice
(19:30:51) Didleth: i belive i have a problem to go to the program from the comandline
(19:31:37) txopi: down
(19:31:39) txopi: enter
(19:32:15) Didleth: :-/ can i just give suto ninjahelper restart ?
(19:32:22) Didleth: *sudo
(19:32:27) Alster: this doesnt exist
(19:32:33) Alster: ninjahelper is not a service
(19:32:40) txopi: ninjahelper is a program
(19:32:50) txopi: like vi or jed
(19:32:58) Alster: yes, it's just this text graphics interface
(19:33:05) Didleth: :-/ ok.... so i should put ok?
(19:33:17) Alster: you could now create a new system info handler, then return to the main menu, then press cancel, then restart ninjahelper an check if the new system info handler is still there.
(19:33:29) txopi: the focus is over option 1 (systeminfo)
(19:33:45) txopi: you want to run 20.pgsql
(19:33:50) Alster: i need a quick bathroom visit, 3 minutes
(19:33:52) txopi: so don't push enter
(19:34:06) txopi: push down and then, when the focus is over option 2, oush enter
(19:34:11) txopi: push down and then, when the focus is over option 2, push enter
(19:34:19) Didleth: /etc/backup.d/20.pgsql ?
(19:34:26) Alster: thanks for continueing in the meantime txopi
(19:34:29) txopi: yes
(19:34:33) txopi: np Alster
(19:35:14) txopi: as you see the handler you created is still there after quiting and running ninjahelper again
(19:35:34) txopi: ninjahelper creates files and change the config files for you
(19:36:11) txopi: now the idea was to execute a test run, but i'm not sure if Alster will want to do that
(19:36:26) txopi: i propose to you something different Didleth
(19:36:27) Didleth: so we wait for him
(19:36:34) Didleth: yes?
(19:36:46) txopi: you created a backup handler for all postgresql
(19:37:00) Didleth: :-/ sorry
(19:37:08) txopi: and i created another handler just for indyeh's database
(19:37:09) Didleth: i didn't wanted to do this
(19:37:11) txopi: no, no problem
(19:37:24) txopi: we are just playing with ninjahelper
(19:37:39) txopi: i propose to you to delete the handler for indyeh
(19:37:57) txopi: because it has no sense to backup all postgresql and the backup again one database
(19:38:01) Didleth: delete? you wante all basque will kill me? ;>
(19:38:09) txopi: do you think you are able to do it?
(19:38:09) Didleth: hehe, ok
(19:38:16) txopi: ok
(19:38:17) Didleth: i donty know
(19:38:21) txopi: try
(19:38:25) Didleth: but i will never know if i will not try
(19:38:40) txopi: that's it :-)
(19:38:51) ***Alster back
(19:39:15) txopi: enter
(19:39:17) Didleth: itd the 3 in the main menu, right?
(19:39:22) txopi: right
(19:39:31) Didleth: kill?
(19:39:36) txopi: yes
(19:39:52) txopi: perfect!
(19:39:54) Didleth: done
(19:40:11) txopi: congratulations Didleth
(19:40:14) Didleth: thx :]
(19:40:16) txopi: you learn really quick!
(19:40:31) Didleth: ok, than we can try to make test alster?
(19:40:35) txopi: Alster, do you want to make a test run of 20.pgsql?
(19:41:05) Alster: hmm can i see your configuration first?
(19:41:34) Alster: i'll just drive quickly
(19:41:51) Didleth: ok
(19:42:21) Alster: ok so this is the configuration of the one postgresql handler we have
(19:42:38) Alster: it dumps all databases and then compresses this dump
(19:42:46) Alster: this is normally good
(19:43:21) Alster: but what happens when you try to run diff on two compressed files which contain similar data?
(19:44:29) Alster: for a reminder: rdiff-backup, the tool we use for making backups runs something similar to the 'diff' command on all the files it backs up, so that it can determine what has changed in these files
(19:44:45) Alster: only the changes it detects are then copied to the remote location
(19:45:00) harcesz: szczym is the first techie of imc-pl (AFAIK), I mentioned his server earlier
(19:45:06) Alster: that's an incremental backups. it saves our bandwidth and storage space on the backup server.
(19:45:06) Didleth: Alster: do you thing that backupninja can take them as the one?
(19:45:11) szczym: helo alister
(19:45:15) Didleth: and can make mistake?
(19:45:15) txopi: if copies all the file?
(19:45:34) Alster: harcesz: we met before
(19:45:40) Alster: welcome szczym
(19:45:41) txopi: hi szczym
(19:45:46) szczym: hi all
(19:46:01) Alster: Didleth: BN can take what as what?
(19:46:36) ***szczym hugs everybody with love and spliff
(19:46:43) Didleth: i mean, did you afraid that BN can mistake this files?
(19:46:52) Alster: txopi: probably, at least much more than it would need to if the two files were not compressed
(19:47:01) txopi: if the dump is compressed rdiff-backup will think that all the dump has changed and will copy it it entirely (or almost all)
(19:47:15) Alster: exactly
(19:47:29) txopi: but if we don't compress it, the first time will need more time but the incremental backup will be quicker
(19:47:33) txopi: isn't it?
(19:48:00) Alster: Didleth: backupninja will not backup the wrong files or something if the database dumps are compressed. but the backup is less efficient.
(19:48:09) Alster: exactly txopi
(19:48:23) Alster: Didleth: do you understand the rationale yet?
(19:48:33) Didleth: yes, i understand
(19:48:40) Alster: very well :)
(19:48:53) Didleth: i'm not sdure or i good understand your question
(19:49:06) Alster: which one?
(19:49:31) Didleth: but i understood what txopi said and i can still thing rational , so for both sence i understand the answer is 'yes' ;-)
(19:49:48) Alster: ok, thanks all we need :)
(19:49:50) Alster: so i suggest we do not compress the database dumps
(19:49:53) txopi: :-D
(19:49:56) Didleth: i agree
(19:49:59) txopi: ok
(19:50:23) Alster: my question was more complex than neccessary again and txopi explained it better.
(19:50:36) Alster: so i'll deactivate the compression
(19:51:15) Didleth: yhm
(19:51:53) txopi: ok
(19:52:21) Alster: i'm not sure this naming makes sense, since it will be incorrect if the configuraition but not the name is changed
(19:52:49) Alster: that's better
(19:53:07) Alster: so, we could now to a test run of the database dump
(19:53:15) txopi: ok
(19:53:31) Alster: it will take quite some time, but we can continue working in another screen window
(19:53:54) Alster: who volunteers running the test?
(19:53:59) txopi: me!
(19:54:07) Alster: \o/
(19:54:17) Alster: give it a go
(19:54:44) txopi: i opened another screen
(19:54:48) Didleth: shoud we join to another screen?
(19:55:09) Alster: txopi: i meant another virtual window in the same screen session
(19:55:17) txopi: mmmh
(19:55:28) Alster: you can switch between windows using ctrl-a-NUMBER
(19:55:32) txopi: i just executed "# screen"
(19:55:39) txopi: aaah
(19:56:18) Alster: please everyone do this:
(19:56:52) Alster: hold 'Ctrl' and press 'a' once, then press '1' once and release 'Ctrl'
(19:57:14) Alster: you should then see a different screen
(19:57:28) Didleth: i dont see
(19:57:29) Alster: where txopi is already preparing the test run
(19:57:30) txopi: i am ready to execute the test run
(19:57:42) Didleth: can i quit from ninjahelper"
(19:57:44) Didleth: ?
(19:57:48) Didleth: and than try?
(19:57:49) txopi: no
(19:57:57) txopi: well, if yoo want
(19:58:04) Alster: well in window 0 you can quit backupninja
(19:58:17) Alster: window 0 is where we have been working so far
(19:58:32) Didleth: yes that what i mean - i belive backupninja i a reason thatwhy ctrl+a+1 doenst work
(19:58:47) Alster: no it is not the reason
(19:58:54) Didleth: blahm romoved from kosmos
(19:59:03) Didleth: i have to join again
(19:59:42) txopi: Alster, let quit from window 1 and repeat the operation when Didleth joins
(19:59:46) txopi: ok?
(19:59:49) Alster: ok
(20:00:05) Alster: the real reason is that my instructions were wrong
(20:00:13) Didleth: ok i see
(20:00:15) Alster: <Alster> hold 'Ctrl' and press 'a' once, then press '1' once and release 'Ctrl'
(20:00:18) Alster: ^^wrong
(20:00:21) Didleth: this is window number 1
(20:00:31) Alster: vv right
(20:00:50) Alster: hold 'Ctrl' and press 'a' once, release 'Ctrl' then press '1' once
(20:01:38) Didleth: kosmos:~# # this is window number 1
(20:01:47) Didleth: am i on a good window?
(20:01:48) Alster: so Didleth is looking at window #1
(20:02:02) txopi: i don't know where i am...
(20:02:07) txopi: wait a bit please
(20:02:21) txopi: ah ok
(20:02:25) txopi: i am ready
(20:02:50) txopi: i'm going to run ninjahelper at window #1
(20:03:21) Alster: ok, I will quit ninjahelper in window 0 then to make it more obvious
(20:03:47) txopi: do i execute test run?
(20:03:57) Alster: you're welcome to
(20:04:19) Didleth: wait txopi
(20:04:25) txopi: ok
(20:04:48) Alster: hmm that's a buit quick, isn't it
(20:05:13) Didleth: ok i see
(20:05:18) txopi: yes
(20:05:23) Didleth: i only wanted to read what ninjahelper write about this
(20:05:27) Didleth: you can continue
(20:05:53) Alster: please stay there for now
(20:06:00) Alster: so we can look at the output
(20:06:32) Alster: what ninjahelper (NH) did there is that it tried to create the dump
(20:06:54) Alster: it should have taken longer, i don't know why it took so short, something is apparently wrong and it failed to inform us about it
(20:07:11) Alster: so that's a bug in this version of NH/BN
(20:07:41) Alster: i have filed a bug against it which can be related and intrigeri, one of the developers, fixed it the other day
(20:08:07) Alster: so it might be a good idea to update backupninja later
(20:08:18) Alster: for now, let's have a look at the output in detail
(20:08:18) Didleth: yhm
(20:08:43) Alster: every line is prefixed with 'Debug: ' so the output is extra verbose apparently
(20:08:59) Alster: it would not normally provide this much output, it is noly done for testing now
(20:09:01) txopi: aha
(20:09:44) txopi: tail -f /var/log/postgresql/postgresql-8.3-main.log
(20:09:45) txopi: WARNING: nonstandard use of \' in a string literal at character 2093
(20:09:45) txopi: HINT: Use '' to write quotes in strings, or use the escape string syntax (E'...').
(20:10:04) ***txopi is exploring logs world...
(20:10:08) Alster: these tests are actualyl run by BN, or rather the BN handlers, NH just invokes BN for it
(20:10:25) Didleth: i dont understand :-/
(20:10:31) Alster: good tyopi, we'll get to this shortly
(20:10:44) Didleth: whats exactly wrong txopi?
(20:10:48) txopi: i understand what Alster said
(20:10:54) Alster: we'll see this in a minute Didleth
(20:10:55) txopi: i don't kwno
(20:11:10) Alster: let'S focus on the output on screen for now
(20:11:14) txopi: i just dump here what could be the reason of the problem
(20:11:21) Alster: in the beginning BN runs some safety checks
(20:11:26) txopi: to understand it is too much for me
(20:11:55) Alster: it checks whether it can actually write to the local directory where the database dumps are to be created
(20:11:58) Didleth: whel, you understand that its something wrong in the tail-results; its more that i understand
(20:11:58) txopi: personally i think that it can be a wrong SQL sentence (bad syntax) when trying to split a long sql
(20:13:06) Alster: and whether nobody else may read files in this directory, since database dumps are a sensitive matter, they often contain passwords (such as your IMC's Mir admin passwords) in plain text or another recoverable way.
(20:13:29) txopi: Alster, i understand what you say about checks
(20:13:32) Alster: it is then happy with the permissions and continues invoking the pgsql handler
(20:14:03) Alster: this again ensures permissions are set correctly, then ries to do the actual DB dump in this line:
(20:14:07) Alster: Debug: su - postgres -c "/usr/bin/pg_dumpall > /var/backups/postgres/kosmos-all.sql"
(20:14:35) Alster: it then _incorrectly_ reports that this action was successful:
(20:14:39) Alster: Info: <<<< finished action /etc/backup.d/20_dump-databases.pgsql: SUCCESS
(20:14:58) txopi: aha
(20:15:07) Alster: it finally sends an email to root, i.e., all of us, to inform us of the supposedly successfull operation
(20:15:19) Alster: that's it, "FINISHED"
(20:15:35) Alster: unfortunately the dump did not really succeed, and now we need to find our why
(20:16:04) Alster: we can do so by inspecting log files of the commands which we assume have failed
(20:16:32) Alster: another way is to repeat running the commands which failed manually, possibly with extra options to get a more verbose output
(20:17:03) Alster: as i said, the command(s) we believe have failed is this: su - postgres -c "/usr/bin/pg_dumpall > /var/backups/postgres/kosmos-all.sql"
(20:17:18) Alster: we can now run this command manually in another screen window
(20:17:32) Alster: let's switch to window 0: Ctrl-a then 0
(20:17:42) Didleth: i belive some try to go to another window ;]
(20:18:06) Alster: you should see this on the bottom of the other window: kosmos:~# # this is window 0
(20:18:24) txopi: i am at window 0
(20:18:28) Didleth: i see
(20:18:56) Alster: Didleth: do you see this text?
(20:19:02) Didleth: yes
(20:19:05) Alster: '# this is window 0'
(20:19:09) Alster: ok
(20:19:37) Alster: i guess i should explain what this command does
(20:20:02) txopi: it would be good
(20:20:16) Alster: 'su -' is a command which ensures that what follows it is run by user 'root'
(20:20:29) Didleth: gives results of the permissions to the db dumb?:-/
(20:20:38) txopi: i know that
(20:20:48) Didleth: i know 'su' too
(20:21:04) Alster: now we are already user 'root' so we could omit it, but ninjahelper may also be run by normal users like you and me so it prefixes this by 'su -' to make sure
(20:21:34) Alster: so the remaining part which is run as user 'root' is: postgres -c "/usr/bin/pg_dumpall > /var/backups/postgres/kosmos-all.sql"
(20:21:58) Alster: postgres is a program to work with the database
(20:22:15) Alster: oh crap i'm explaining it all wrong...
(20:22:25) Alster: ok, back to the whole command
(20:22:30) Alster: su - postgres -c "/usr/bin/pg_dumpall > /var/backups/postgres/kosmos-all.sql"
(20:23:10) Alster: this is actually this command 'su - postgres -c' invoking this command '/usr/bin/pg_dumpall > /var/backups/postgres/kosmos-all.sql'
(20:23:36) Alster: 'su - postgres -c' makes what follows it run as user 'postgres'
(20:23:54) Didleth: :-/
(20:23:59) Alster: so '/usr/bin/pg_dumpall > /var/backups/postgres/kosmos-all.sql' is run as user 'postgres'
(20:24:03) txopi: tlf
(20:24:16) Alster: sorry for the confusion i caused by explaining it wrong at first
(20:24:30) Didleth: /usr/bin/pg_dumpall > /var/backups/postgres/kosmos-all.sql' it redirect /usr/bin/pg_dumpall to /var/backups/postgres/kosmos-all.sql' ?
(20:24:44) Alster: not exactly.
(20:24:48) Alster: /usr/bin/pg_dumpall is a command
(20:24:49) Didleth: :-/
(20:24:54) Alster: you could run it directly
(20:25:44) Alster: but to make sure this command is not run by root or any other user but only by the user named 'postgres' (there is such a user on kosmos) it is prefixed by 'su - postgres -c'
(20:26:21) Alster: so 'su - postgres -c' only says "run whatever follows me as a command on its own, but run it as user 'postgres'"
(20:26:24) Didleth: yhm, i understand - and the results of the comand are writen in /var/backups/postgres/kosmos-all.sql'?
(20:27:17) Alster: that's right, the output (standard output only, no errors) generated by /usr/bin/pg_dumpall is written to /var/backups/postgres/kosmos-all.sql
(20:27:57) Didleth: so we should now look into /var/backups/postgres/kosmos-all.sql ?:-/
(20:27:57) Alster: and /usr/bin/pg_dumpall is supposed to do what the name indicates: it should dump all databases of our postgresql database cluster
(20:28:07) Alster: that'S a good idea
(20:28:26) Alster: bad luck, it doesn't exist
(20:28:38) Alster: probably because the command failed somehow
(20:28:39) Didleth: :-/
(20:28:53) Alster: lets just run the command now
(20:29:07) Didleth: can we make this coamnd step by step maybe?
(20:29:12) Alster: hmm this takes a long time now. i guess it does work this time
(20:29:30) Didleth: so first log as postgres and check or there is everything ok with this users?
(20:30:03) Didleth: yhm, so the file doesnt exist becouse the oporetion wasnt finishet yet?
(20:30:17) Alster: yes we could do this, but since it is already woring now it doesn't seem to be neccessary
(20:31:17) Alster: the database dump at /var/backups/postgres/kosmos-all.sql did not exist since the operation failed when it was invoked by BN.
(20:31:41) Alster: it doesn't sem to be failing now that we run it as root
(20:32:11) Alster: i will now open yet another scree window, window #2
(20:32:22) txopi: i was talking by telephone
(20:32:23) Alster: to check whether /var/backups/postgres/kosmos-all.sql is growing in size
(20:32:29) txopi: i follow you
(20:32:53) txopi: in half an hour i will have dinner
(20:33:07) Alster: ok
(20:33:14) Alster: so the fil is growing in size
(20:33:38) Alster: so we can assume the database dump is actually working now
(20:34:11) Alster: now what we need to understand is why it works now but did not work when BN invoked it.
(20:34:18) Alster: it's the same command after all
(20:35:01) Alster: my _guess_ is that backupninja just pretended to invoke the command but did not really do it since we are just in testing mode
(20:35:39) Alster: this would mean it only checks whether the permissions are ok but does not actually start the database dump process in testing mode
(20:36:08) Alster: we would have to really run handler to make it work
(20:36:34) Didleth: alster: andy have bacupninja all persmissions?
(20:37:08) Alster: Didleth: it should, since we ran it as root. that is, unless NH ran it as another user
(20:37:14) dez`afk nazywa się teraz Dez
(20:37:16) Alster: but i think it actually runs as root
(20:37:30) Didleth: o dez is come back
(20:37:35) Alster: i'll cancel the running dump in window #0 now
(20:37:44) Didleth: dez change nick, its logged
(20:37:54) Alster: ^Cpg_dumpall: pg_dump failed on database "eh_indy", exiting
(20:37:57) Didleth: and hi
(20:38:39) Alster: i just verified that backupninja is normally run as user root
(20:39:01) Alster: ..when it is invoked by cron to make backups
(20:39:10) txopi: Alster, i think you are right! in a test run ninjahelper just check permissions and show you what ninjabackup will do!
(20:39:26) Alster: i hope so, too :)
(20:39:26) Didleth: so maybe just run something not in test-mode
(20:39:35) Didleth: and vergleichen?
(20:40:03) Didleth: btw i ont see nothing new on screen
(20:40:06) Alster: i did run postgresql dumps with this version of backupninja on kosmos before so it should work indeed
(20:40:21) Alster: bach to window #1
(20:40:31) Alster: ..where we used to run the test
(20:40:54) Didleth: ok it works
(20:40:56) ***txopi is at window 1
(20:40:59) Dez: hi all, hi Didleth
(20:41:22) txopi: hi dez
(20:41:38) Alster: ah right we can choose between 'test' and 'run' there
(20:41:39) ***txopi is from basque country
(20:41:40) Alster: hi dez
(20:41:46) Dez: I'm back from 2 weeks offline. I'll come to life quite slowly but I'll do :)
(20:41:59) Didleth: :]
(20:42:02) Dez: ( hi txopi, Alster )
(20:42:06) Alster: 'test' just does the permissions check, and that'S what we just did. 'run' will actually run the database dump.
(20:42:27) txopi: aha
(20:42:58) Alster: so i'll really 'run' it this time
(20:43:06) Alster: so that we can know for sure whether it works
(20:43:12) txopi: just do it! :-)
(20:43:18) Didleth: i agree :]
(20:43:19) Alster: we can still cancel it
(20:43:31) Alster: ok, this looks like it should be looking
(20:43:40) txopi: yes :-)
(20:44:01) Alster: can you switch to window #2 please
(20:44:12) txopi: done
(20:44:19) Didleth: ctrl+a, 2?
(20:44:22) Alster: yes
(20:44:33) Didleth: done
(20:45:00) Alster: i'll run 'head' on the database dump so we can see some of the files' contents
(20:45:36) Alster: so, as you can see, at least this part of the file consists of plain text
(20:45:50) Alster: it's not like cryptic stuff which is unreadable to humans
(20:46:00) txopi: aha
(20:46:05) Didleth: yhm
(20:46:11) txopi: it isn't compressed
(20:46:51) Alster: compare this to this
(20:46:52) Didleth: its not A PLAIN TEXT
(20:46:54) Didleth: AS I SEE
(20:46:56) Alster: doesn't look so nice
(20:47:01) txopi: hehe
(20:47:03) Didleth: blah, sorry, caps lock again
(20:47:43) Alster: actually the second file would be fine for rdiff-back to work with. too, if the contents would not change so much
(20:48:10) Alster: nut they will change much if it is a gzip file format
(20:49:01) Alster: but if it's just text and just some part of the text changes (since only some part of the database contents changed) it is much better
(20:49:17) Alster: ok, i think this is as clear as it gets by now
(20:49:24) txopi: yes
(20:49:27) txopi: so, what next
(20:49:40) txopi: should now talk about next meetign, etc.?
(20:49:41) Alster: unfortunately the uncompressed backups are larger, so we need to ensure we have enough space left
(20:49:51) Alster: /dev/md4 135G 132G 2.7G 99% /var
(20:49:55) txopi: ups, sorry
(20:50:07) Alster: i guess tonight's database dump may already fail with this
(20:50:25) txopi: :-(
(20:50:38) Alster: this recurring problem is caused by the other database dump script
(20:50:48) Alster: since it fails to 'rotate' the dumps
(20:51:02) Alster: it keeps creating more and more but never deletes any automatically as needed
(20:51:18) Alster: you know this since the partition keeps running full
(20:51:27) Alster: we need to fix this soon
(20:52:10) txopi: aha
(20:52:15) Alster: either by making the script delete old locally stored database dumps before creating new ones, or by deactiating it altogether since we (hopefully) have working remote database backups now/soon
(20:52:30) Alster: i think the first option is actually preferrable, at least for now
(20:53:17) txopi: deactiating?
(20:53:24) Alster: backupninja will always back up to the same file location, namely /var/backups/postgres/kosmos-all.sql so this file always gets overwritten and this is not an option for storing local 'backups'
(20:53:53) Alster: deactiating -> deactiVating / disabling / removing / killing
(20:54:03) Alster: this was a typo there, sorry
(20:54:27) Alster: for now, i suggest we just delete some of the old database dumps as usually
(20:54:28) txopi: so you mean deactivating what? the script?
(20:54:45) txopi: Alster: ok
(20:55:05) txopi: Didleth, is a crack doing that! :-D
(20:55:19) Alster: either by making the old database dump script delete old locally stored database dumps before creating new ones, or by removing the old database dump script altogether since we (hopefully) have working remote database backups now/soon
(20:55:27) Alster: ^^ this is what i meant
(20:56:10) txopi: so you propose to change the script, not deleting manually?
(20:56:21) Didleth: you menaed i forgot to delete old backups?
(20:56:34) Alster: and i think having some compressed local copies of the database is a good idea (though a few days of backlog should suffice, we don't really need > 1 week)
(20:57:21) txopi: gdm or someone who put that script on kosmos tryed to fix it but finally didn't do. so i think it is a complex problem
(20:57:24) Alster: i propose to modify or extend the old database dump script so that, every time it runs, it deletes some old dumps before it creates new ones
(20:58:00) Alster: it's not that complex, but it may require some bash scripting, which requires some bash scripting experience.
(20:58:16) txopi: aha
(20:58:26) Alster: maybe the old database dump script even provides an option to do just this, i have not checked.
(20:58:53) txopi: people is waiting to me for dinner :-)
(20:58:55) Alster: oops
(20:59:03) Alster: we have a database dump error
(20:59:08) Didleth: so, and for today?
(20:59:17) Alster: check window #1
(20:59:20) Didleth: txopi has to go?
(20:59:23) Didleth: ok
(20:59:33) Alster: ok,have a nice supper txopi
(20:59:41) Alster: Warning: pg_dump: dumpBlobs(): could not open large object: ERROR: large object 250866 does not exist pg_dumpall: pg_dump failed on database "eh_indy", exiting
(20:59:41) Alster: Warning: Failed to dump pgsql cluster
(20:59:41) Alster: Warning: <<<< finished action /etc/backup.d/20_dump-databases.pgsql: WARNING
(20:59:49) Didleth: ctr+a, 1 doesnt work
(21:00:00) Alster: try again please
(21:00:54) Didleth: blah, whats a lag
(21:00:57) Alster: maybe your connection is just stalled again
(21:01:07) Didleth: i hope not spoilt something trying go to view !
(21:01:08) Didleth: 1
(21:01:15) Didleth: maybe
(21:01:22) Alster: nah its all still there
(21:02:02) Alster: i have not seen this error message before so i can just guess what caused it
(21:02:13) Alster: or how to recover from it
(21:02:45) txopi: mmmh
(21:02:51) Didleth: so...what is yours guess?
(21:02:59) txopi: i have no idea about the reason of the error
(21:03:42) Didleth: maybe
(21:03:50) Didleth: something with badsectors?
(21:03:58) Alster: i assume that there was something accessing the database at the same time the dump process was accessing it
(21:04:04) txopi: if the usual script is working, perhaps we could take a look and check how it does. perhaps we see something different that can help us
(21:04:27) Alster: it could also mean that there is a persitent inconsistency in the database
(21:04:48) txopi: perhaps we should stop postgresql and the make the dump
(21:04:53) Didleth: we make this same twice, once maualy and once by ninjahelper, so that could ber the accesing Alster is wrote about
(21:05:30) txopi: if not, it has to be very difficult to make a database dump and write it all consistent while there could be some transactions running
(21:05:50) Alster: i would expect the backupninja postgresql handler to use table locking which should prevent concurrent accesses to the database and thus make the dump work without such errors.
(21:06:19) txopi: sure? i don't know
(21:06:29) Alster: that's why i assume it's rather a persistent database inconsistency
(21:07:21) txopi: so bad news, specially for eh_indy...
(21:07:25) Alster: it doesn't hurt to check the DB for inconsitencies (and have them repaired if any)
(21:07:51) Didleth: txopi: why bad news?
(21:07:54) txopi: if there is a persistent error, will the local backups be working??
(21:07:55) Alster: databases are really good in recovering from inconsistencies, about as good as jorunalling file systems
(21:08:30) Alster: if there is a persistent inconsistency the other dumps may have failed since it first occurred, too
(21:08:43) txopi: i'm very sorry but i have to go now
(21:08:55) txopi: if you continue i will read you on the logs
(21:08:56) Alster: so it can be good idea not to delete all the old dumps right now
(21:09:02) ***Didleth doesnt know what is persistent error
(21:09:06) txopi: anyway, see you on tuesday ok?
(21:09:21) Alster: Didleth: one that remains (until it gets fixed)
(21:09:24) txopi: good bye
(21:09:30) Alster: bye bye
(21:09:36) Didleth: bye txopi
(21:10:00) Alster: so i'll look up the tools to check the database for consistency
(21:10:07) Didleth: yhm
(21:10:08) Alster: since i don't remember them myself
(21:16:27) Alster: i'm working in window #0
(21:16:51) Alster: i will stop tomcat to make sure nothing else is accessing the database right now
(21:17:01) Didleth: yhm
(21:17:17) Alster: this means admin/posting/search won't work in the meantime, of course
(21:17:31) Didleth: yes, i know
(21:18:03) Alster: i started the analysis, bt then cancelled it since it was providing too uch output
(21:18:41) Didleth: what is pg mainteanace?
(21:19:35) Alster: it is a database maintennance script
(21:20:02) Alster: it can be used to defragment and to analyse the database
(21:20:30) Alster: i _think_ it can also fix incinsistencies it finds while processing it.
(21:21:11) Didleth: yhm, should i go to window 2?
(21:21:21) Alster: yes might be a good idea
(21:21:36) Alster: i'm running 'tail' on the database there now
(21:22:21) Didleth: blah sorry i put yhm in bad window, did i broke up something?
(21:22:42) Alster: by the way, running out of disk space on the file system is a common cause for database inconsistencies
(21:23:11) Alster: no it's fine
(21:23:26) Alster: (you did not break anything)
(21:23:33) Didleth: i have to prepare some linux-worshops for fryday yet :-/
(21:23:45) Didleth: but i can do this in the week
(21:23:48) Alster: you can do it now, this process will take some time
(21:23:55) Didleth: yhm, so ok
(21:25:21) Alster: hmm the log looks like it is done but the process has not exited yet
(21:28:14) Didleth: :-/
(21:38:51) Alster: ok it's still writing to it, i just didnt realize it
(21:42:39) Alster: ok it's done
(21:44:45) Alster: i'll try to make naother backup using the backupninja handler
(21:45:15) Alster: ...only after restarting postgresql
(21:45:21) Didleth: ok
(21:46:07) Alster: you don't need to watch my progress, i'm just reporting back for the backlog ;-)
(21:47:00) Didleth: ok - so i will a little bit rest (i'm a little bit ill)
(21:47:06) Didleth: till later
(21:47:36) Alster: get well soon then, and see you later
(22:00:49) Alster: i need to make some room, so i'm deleting the database dumps of all uneven days between Dec 10 and Dec 31:
(22:00:52) Alster: kosmos:~# du -sch /var/backup/December-2009/2009-12-*{1,3,5,7,9}
(22:00:52) Alster: 3.5G /var/backup/December-2009/2009-12-11
(22:00:52) Alster: 3.5G /var/backup/December-2009/2009-12-21
(22:00:52) Alster: 3.5G /var/backup/December-2009/2009-12-31
(22:00:52) Alster: 3.5G /var/backup/December-2009/2009-12-13
(22:00:52) Alster: 3.5G /var/backup/December-2009/2009-12-23
(22:00:55) Alster: 3.5G /var/backup/December-2009/2009-12-15
(22:00:56) Alster: 3.5G /var/backup/December-2009/2009-12-25
(22:00:58) Alster: 3.5G /var/backup/December-2009/2009-12-17
(22:01:00) Alster: 3.5G /var/backup/December-2009/2009-12-27
(22:01:02) Alster: 3.5G /var/backup/December-2009/2009-12-19
(22:01:05) Alster: 1.5G /var/backup/December-2009/2009-12-29
(22:01:06) Alster: 37G total
(22:01:10) Alster: kosmos:~# rm -rf /var/backup/December-2009/2009-12-*{1,3,5,7,9}
(22:01:25) Alster: kosmos:~#
(22:01:52) Alster: kosmos:~# df -h /var
(22:01:52) Alster: Filesystem Size Used Avail Use% Mounted on
(22:01:52) Alster: /dev/md4 135G 92G 43G 69% /var
(22:01:52) Alster: kosmos:~#
(22:01:57) Alster: much better
(22:03:16) Alster: these database dumps are remaining:
(22:03:19) Alster: kosmos:~# du -sch /var/backup/{November,December,January}*/*/
(22:03:19) Alster: du: cannot access `/var/backup/November*/*/': No such file or directory
(22:03:19) Alster: 3.5G /var/backup/December-2009/2009-12-12/
(22:03:19) Alster: 3.5G /var/backup/December-2009/2009-12-14/
(22:03:19) Alster: 3.5G /var/backup/December-2009/2009-12-16/
(22:03:19) Alster: 3.5G /var/backup/December-2009/2009-12-18/
(22:03:22) Alster: 3.5G /var/backup/December-2009/2009-12-20/
(22:03:22) Alster: 3.5G /var/backup/December-2009/2009-12-22/
(22:03:25) Alster: 3.5G /var/backup/December-2009/2009-12-24/
(22:03:28) Alster: 3.5G /var/backup/December-2009/2009-12-26/
(22:03:29) Alster: 3.5G /var/backup/December-2009/2009-12-28/
(22:03:30) Alster: 4.0K /var/backup/December-2009/2009-12-30/
(22:03:34) Alster: 3.5G /var/backup/January-2010/2010-01-01/
(22:03:36) Alster: 7.4M /var/backup/January-2010/2010-01-02/
(22:03:38) Alster: 3.5G /var/backup/January-2010/2010-01-03/
(22:03:41) Alster: 2.3G /var/backup/January-2010/2010-01-04/
(22:03:44) Alster: 3.5G /var/backup/January-2010/2010-01-05/
(22:03:44) Alster: 3.5G /var/backup/January-2010/2010-01-06/
(22:03:48) Alster: 3.5G /var/backup/January-2010/2010-01-07/
(22:03:48) Alster: 3.5G /var/backup/January-2010/2010-01-08/
(22:03:51) Alster: 3.5G /var/backup/January-2010/2010-01-09/
(22:03:54) Alster: 3.5G /var/backup/January-2010/2010-01-10/
(22:03:54) Alster: 61G total
(22:03:57) Alster: kosmos:~#
(22:04:08) Alster: should suffice to roll something back in case of error
(22:04:38) Alster: the dump from 2010-01-02 is pretty small. i guess the disk may have run full that day
(22:04:58) Alster: 2010-01-04 is too small, too
(22:05:20) Alster: they will be incomplete so i'm deleting them, too
(22:12:59) Alster: i'm moving old log files of the postgresql 7.4 cluster which are no longer in use out of the way, from /var/log/postgresql/ to /var/backup/zapata_and_alster/pg7.4to8.3/postgresql74-logfiles.tar.gz
(22:15:41) Alster: looks like i lost my connection to kosmos ... good this stuff is running in screen
(22:23:38) Alster: actually it returned
(22:23:56) Alster: dump is still going
(23:10:41) Alster: it's still dumping...
(23:11:00) Alster: it's definately taking longer than last time now
(23:11:42) Didleth: yhm
(23:11:46) Alster: so I assume running pg_maintence did help and this running dump process can succeed
(23:12:23) Alster: the dump is 12GB already
(23:12:38) Alster: but this was to be expected
(23:14:12) Alster: the binary data stored in /var/lib/postgresql/ is only 5.1GB though
(23:36:42) Alster: ok, the dump is done and looks good, 13G total
(23:36:58) Alster: backupninja is happy, too
(23:41:42) Alster: Since the backup is stored at /var/backups/postgres/kosmos-all.sql and this location is already covered by the rdiff-backup handler for jordie ("include = /var/backups", no symbolic links in this path), the first dump should automatically get included in the backup to jordie tonight.
(23:42:25) Alster: i've started tomcat again.
(23:43:21) Alster: the posting form seems to be working
(23:43:42) Alster: (the one of PL that is)
(23:44:53) Alster: so we need yet to fix the old postgresql dump script so that it rotates backups, but at least we should have a complte remote backup by tomorrow. this is something we should verify yet.
(00:06:36) Alster: Script done, file is skill-sharing-session_2010-01-10_kosmos.script
skill-sharing-session-10-log
This site is run by riseup.net, your friendly autonomous tech collective since 1999.
This site is powered by crabgrass, AGPL software libre for network organizing