Yesterday I was thinking about genealogical interchange formats, because Beyond will need to send diffs back and forth (between the server and client). GEDCOM is the current standard, but it’s old (the specification was written in 1996). Enter XML. It’s flexible, very well-supported, and it’s nice to work with. Sticking with GEDCOM is like using PCX files (remember those?) instead of anti-aliased PNGs. Not good.

So, there are a few XML standards out there, but none have replaced GEDCOM yet. Why? I don’t know. But it’s time for a new standard to emerge.

That said, what needs to happen is close inspection of the existing standards to find out where they succeed and (more important) where they fail. And it wouldn’t hurt to examine other standards that have taken off like wildfire (RSS, PNG, etc.) and see what made them work. Is it just application support? Is it accompanying tutorials and articles? Is it name recognition? Hmm…

Technorati Tags: genealogy, GEDCOM, XML

The site has moved to www.beyondproject.org. The old URL still works, but I recommend moving to the new one. :)

Anyway, I just finished finals and am starting to delve into Ruby on Rails. More to come…

Technorati Tags: Ruby on Rails

This morning I discovered Pixoh, an online photo editor. Quite cool. But what it made me think of was this: a similar kind of site but for genealogy. It’d be the Beyond web client but allow you to open a local file from your hard drive or flash drive (in Beyond format, PAF, GEDCOM, whatever), work on it, and then save it again to a local file. That way you wouldn’t have to have a hosting server to use Beyond online. You’d lose all the advantages of the genealogy-anywhere model, of course, and you’d have to store the file yourself, but it could be nice if you don’t want to get hosting. There’d have to be strict privacy, of course (no storing the files on the server after you close the window, or after 24 hours like Pixoh). Hmm… Any thoughts? Would this be useful? (All this would be in addition to what I’ve already mentioned for Beyond, by the way, not instead of it.)

Technorati Tags: genealogy, Pixoh

[I wrote this up in Writely this afternoon, and it’s proven to be a good consolidation of the various thoughts I’ve had. Feel free to leave comments. As for why it’s a “proposal,” I have to get the project approved by the advisory committee in order to be able to work on it 20 hours a week and get paid for it, so this is what I’m submitting. –Ben]

Beyond Proposal

In a nutshell, Beyond is a cross-platform, open, free genealogy record manager that makes genealogy research easy and fun.

Principle: You should be able to access your genealogy anywhere in the world.

Solution: Through the “Genealogy Anywhere” model, you can. Upload your data to a server running Beyond, and from then on you’ll be able to get to your genealogy from any computer in the world using the web client. What if you want to work on your genealogy midflight en route to Europe? Before you leave, let Beyond automatically download your data to your laptop (using the desktop client). Make whatever changes you want. When you get off the plane and find a spot with Internet access, plug in and Beyond will take care of synchronizing your new changes with the server. No matter where you are in the world, you can get to your latest data. No more hassles with flash drives or e-mailing files or (heaven forbid) floppies. You’ll also be able to work purely offline if you want, like PAF.

• Server/client model, with offline local file access available instead
• Web client
• Desktop clients for Mac OS X, Linux, and Windows
• Compatible with PAF 5, 4.x, 2.3.1, GEDCOM, and a variety of other formats
• Host your genealogy on your own server (like WordPress)
• Open source

Examples of the “access anywhere” idea include Gmail/Hotmail/Yahoo (e-mail), Google Calendar (planner), del.icio.us (bookmarks), Flickr (photo storage), Writely (word processing), Backpack (to-do lists and much more), LibraryThing (book collections).

Principle: Genealogy should be easy for everyone.

Solution: Genealogy should not be as hard and irksome as filing your taxes, and yet that is where the current state of record managers remains. But not for long. Beyond will make genealogy easy and fun, through an intuitive interface designed to get rid of all irritations. It just works. Both new genealogists and expert researchers alike will find Beyond a pleasure to work with, because it helps them get things done. And there won’t be excess baggage that gets in the way. Simplicity is beauty, and beauty makes the user experience joyful and satisfying.

• User-friendly
• Simple
• Easy navigation
• Powerful search
• Internationalization in dozens of languages
• Unlimited undo — no more worries about accidentally deleting something

Examples of user-friendliness: Wordpress (blogging).

Principle: Collaboration using genealogy software should be easy and painless.

Solution: One of the greatest advantages of the Internet is the capability it has brought us for collaboration, and genealogy is an area where collaboration can make miracles happen. E-mailing files back and forth and merging the changes by hand is not the best way to bring about those miracles, however. Through its revision control system, Beyond will make collaboration a piece of cake. You can give access to your data to someone else (or even to just part of your tree), and you’ll easily be able to see any changes they make and decide whether to approve them. Merging has never been easier. And once you’ve got something you like, you’ll be able to publish it easily, either through a live link to a web server (dynamically updated) or PDF. If you’re working on a book, Beyond will let you select parts of your tree and save them as EPS diagrams that you can then insert directly into your Word document. Finally, Beyond will interface with other parts of the web, letting you work with Flickr photostreams, sets of del.icio.us bookmarks, WeRelate pages, and even Family Tree. No man (or woman) is an island.

• Revision control for managing changes
• Publishing via live link
• Publishing via PDF
• EPS diagrams
• Interaction with other websites (Flickr, del.icio.us, WeRelate, etc.)

Examples of revision control: Subversion, CVS, darcs.

Principle: Genealogy software should encourage good research.

Solution: Without evidence to back them up, genealogical conclusions oft remain at the level of mere speculation. Beyond makes citing that evidence easy, avoiding the tedious monotony that often accompanies sources. You can rank information sources according to their credibility — as the source of a birth date, for example, a birth certificate holds more credibility than a death certificate for the same person, because the former was made at the time of birth, whereas the death certificate was often made seventy or eighty years later. Incorporating information like this into the evidence system makes it easy for you to come back to your conclusions later on and see why you made them. Beyond also has a built-in research “notebook,” allowing you to easily classify your notes and log your searches. Finally, non-destructive what-if scenarios let you move things around without actually harming your data, seeing what alternate conclusions would lead to. And there will also be tags (ala Flickr), parallel timelines, and much more.

• Weighted evidence system
• Research notebook/log
• What-if scenarios
• Tags
• Parallel timelines

Examples: Backpack (research notebook), Flickr/del.icio.us (tags)

To make collaboration on the design easier, and as a form of documentation for the project (and mainly because I’ve wanted to create a wiki for a while now :)), I’ve created a wiki for Beyond (using MediaWiki). I’m still not sure just how useful it’ll be for design/development, but its usefulness for documentation is clear.

Here’s an example of how I think it’ll be used: I’ll write up a first draft of a spec within the next day or two, and I’ll post it to this blog and to the wiki at the same time. Hopefully it will elicit comments on the blog, and I’ll then edit the wiki version to incorporate any changes. There may be other uses, and they’ll crop up as time goes on.

Assuming finals don’t hit me too hard, I’ll consolidate all the ideas into a map or spec, deciding what’s important and what’s not. (For an initial version, of course.) Simplicity is key. And I’ve got to figure out the whole revision control system. Once I’ve sorted everything into Now and Later categories, I think I’ll start building a prototype…

Technorati Tags: wiki

My papers are finally done and so I should have more time from now on. (Finals won’t be too hard this semester, thankfully.) So, Saturday afternoon I was decluttering my room, and I discovered all my genealogy research notes from fall 2005, on my Iavicoli ancestors in Chieti, Italy, and my Shanks ancestors in Petersburg, Virginia, and my Bailey ancestors in Anson County, North Carolina. There was a ton of stuff and I didn’t really know where to start or even how to organize it.

Sunday morning in the shower, the problem came to mind again. Immediately I thought of Backpack — it’s how I now organize everything else (my to-do lists, extra notes, etc.). And I realized it would be perfect for genealogy.

It’s kind of like a wiki, in a way, but with a few more structural things added in. The basic unit would be a page (like a web page, or a folder), which you could organize however you want. (Flexibility is key — imposing much of a structure will alienate all those who would prefer a different structure. It’s better to let it flow and fit the contours of each user’s way of working.)

On each page, you’d be able to add notes, to-do items, pictures, and whatever else you’d want. Pretend you’re working on your Jones line. You could create a Jones page and basically throw all your stuff in there, and then later on sort it into subpages with the Indiana line, the Illinois line, the Canada line, etc. If you found a document on a microfilm, you could photocopy it and then scan the copy in and attach it to that page. (You could even create pages for individual people, if you had enough material. The point is, you could organize it however you want.) And if you didn’t want to scan it in, you could still put in a “placeholder” document that would let you know you have a paper copy (”This document is in my Chicago folder,” for example).

As far as the Backpackish features go, the question now is whether it’s best to just integrate with Backpack via the API or to try to do it ourselves. The advantage of Backpack is that it’s already written. The disadvantages are that it costs money for the larger accounts (>5 pages), and it’s not specifically tailored for genealogy. Hmm…

This would also act as a kind of scratchpad for research notes, an inbox where you can put things that you later put in a place that makes sense to you. At the same time, however, it’d be really nice to have a “what-if” system in place that lets you work with your actual data, molding it like silly putty into a variety of key shapes in the hope that it’ll fit the lock. It’d have to use copies of your data, of course, like cloning your ancestors and moving them around to see if any connections arise. If users feel like their data is safe and won’t be lost, and if there’s a good way to “play” (i.e., try new ideas out), then they’ll be able to explore and come up with things that wouldn’t have occurred to them otherwise. That’s the theory, at least. More later.

I also think a research log needs to be integrated into the system. There has to be some way of keeping track of what you’ve already done, so that you don’t keep retracing your steps. In my short-lived fling as a family history major, I often dreaded filling out the research logs for my classes. And then I’d get to the end and have to try to fill it in from memory, which was even less fun. There has to be a better way. I don’t know what the answer to that is, though.

Thoughts?

Technorati Tags: genealogy, Backpack

I have class in a few minutes so this’ll have to be quick. I’ll post more later.

The more I think about it, the more offline synchronization seems necessary. Web-only wouldn’t cut it. There are situations where you wouldn’t have Internet access (or wireless) — if you’re on an airplane, for example, or up in the mountains or in a third-world country where there aren’t many connections and the existing ones are slow and expensive.

I’m going to rewrite the project proposal and make it nice and colorful, with diagrams of how the whole thing is going to work out, and I’ll post a PDF when I do that. (Within the next couple of days, I think.)

Time’s run out. I’ll save my idea on research for my next post later on.

Technorati Tags: genealogy

Sorry for the long lag in posting. And this isn’t even a real post, just a short note saying that the end of the semester is approaching rapidly and so things may be on hold until finals are over.

Going along with my ideas over the past few days, particularly that of focusing primarily on making Beyond a web application (though of course the desktop clients will follow, in their own due time), I’m starting to see Beyond in a newer light. I mentioned a little bit about this in my last post: people are inevitably going to want to do more than just edit their genealogy — they want to publish it. Not everything, of course, but they want to make charts that they can print out, and nowadays they want to publish to the web. I’m thinking something like genealogy translated into a blog metaphor, somehow. In fact, I still don’t know exactly how that’d work out, but I do really like the idea of making Beyond be more than just a record manager. This idea of publishing your family tree (or just part of it, even — you’d be able to specify what and how much) dynamically, with a live link, has taken hold of me. I’ve first got to look at the other apps out there — phpGedView, etc. — and see what’s around… And I’ll think about this some more, too.

Earlier this evening one of my Thai friends called and wanted some help putting together a genealogy website for her family. As I was considering what it would take, I realized that this probably fits very well into Beyond’s landscape (web-based, collaboration, etc.). And yet I don’t want that to be Beyond’s main focus — it’s going to be a record manager, and a darn good one at that. But I don’t think the two goals — managing records and publishing them — are mutually exclusive in any way.

As far as the Thai part goes, internationalization is a big thing. PAF, for example, isn’t translated into Thai, and when I try to enter Thai characters into a name field, it thinks they’re numbers. Everything will be Unicode, of course, and I’ll do my best to make sure things work out for languages with scripts radically different from ours, like Chinese, Hindi, and Arabic. More on this later. (Just out of curiosity, do pedigrees in Arabic flow from right to left? Hmm…)

Well, these thoughts aren’t as clear as I would’ve liked, but hopefully they’ll gel over the next day or two and I’ll figure out just what I’m trying to say. :) In the meantime, thanks to all those who’ve given input so far — it’s very helpful and I really appreciate it. Keep it coming. :)

Technorati Tags: genealogy

All afternoon and evening the thought of making Beyond primarily web-based has impressed itself upon me. The web is the future of software development. Sure, there’ll probably always be desktop apps, but the tide is changing (and has already changed in many ways). E-mail started it (with Gmail, Hotmail, Yahoo, etc.), and we’re seeing it now with word processing (Writely), photo storage (Flickr), bookmarks (del.icio.us), library collections (LibraryThing), to-do lists (Ta-da List), and lots more.

So what does this mean for genealogy? Until today, I’ve been firmly of the opinion that genealogy should be done on the desktop. In retrospect, what was I thinking?!? Genealogy is all about collaboration now — that’s the new world that the Internet has opened up to us. In the past, researchers tucked themselves away in corners of the library, with relatively infrequent collaboration happening through writing letters by post and phone calls. It was good, but now we have e-mail, IM, mashups, and social networks. The possibilities are endless. Take WeRelate, for example — a wiki for genealogy (I talked about it a few days ago). All sorts of things are happening, and there’s a lot more that can happen. Tagging, interfaces with maps and library collections, the ubiquity of web browsers, etc.

So, here’s what I imagine: Beyond becomes like WordPress. Rather than worrying about desktop clients, we’ll focus solely on the web client. You install Beyond on a server and can then work on your genealogy from any computer in the world, just as with WordPress you install it on a server and can then work on your blog from any computer in the world.

I’ve considered starting something up to offer hosting (ala WordPress.com), but that’s business and I rather abhor commercial ventures. But of course I wouldn’t be averse to someone else taking care of that (probably with the accounts free for up to a certain number of individuals and after that $5/month or $10/month or whatever, to cover costs and then eventually make a profit). The Church has Family Tree, but I think there’ll still be people who want to store their genealogy on their own servers (just as with blogs, there are lots of people on Blogger but then lots who break off and host their own blog, like I did).

As I’m thinking about this, it’s starting to feel awfully similar to Family Tree. The main difference I’m seeing now is that you can install the software on your own server (as I just mentioned), and I’m thinking there will be several functionality differences as well. But without knowing just what Family Tree is or is planning to become, it’s hard to say.

I feel good about this. Truth be told, I haven’t done much desktop coding in the past five or six years — it’s almost all been web development. This is where I feel at home, and it makes a lot more sense to start here. (Not to mention the fact that this decision makes the whole project a heck of a lot less daunting.) The only disadvantage I can think of is if you go somewhere that doesn’t have Internet access. But really, almost everywhere does, and within the next ten years the number of places on the globe without Internet access may be able to be counted on a single hand. (Global wireless — that would be nice!) Besides, I plan to open an API, which means anyone who wants to write a desktop client (for Mac, Windows, Linux, their cell phone, whatever) could do it easily and have it sync with the server. (I still plan on using a CVS model, by the way.)

Having just come up with this, I haven’t figured out all the details yet, but I think I’ll use Ruby on Rails (Ajaxified), based on an SQL database. Any comments?

One last benefit: if Beyond is first and foremost a web application, integration with other sites becomes natural. Imagine being able to link into your Flickr photos, or get to your genealogy-related bookmarks from del.icio.us, or a WordPress plugin which lists the surnames you’ve been working on recently in the sidebar, or direct links to WeRelate surname/place pages from your pedigree. (This is just brainstorming, by the way.)

Well, this seems to be a good development. It’ll still be a massive task, but not quite as hard as the task of developing both a web client and a desktop client. Things are looking good…

Technorati Tags: genealogy, Gmail, Writely, Flickr, del.icio.us, LibraryThing, Ta-da List, WeRelate, WordPress, Family Tree, Ruby on Rails, Ajax, SQL