Open Scriptures

I have noticed in the Open community that we lack efficient communication and project visibility. Maybe communication is just difficult to begin with, and I bet we do better than purely proprietary top-secret enterprises. But in any case I’ve made the mistake in the past of setting out in isolation to work on some exciting, worthwhile project, but once I stopped and looked around at the community, I’d likely find others around me who are actively working on or who have already completed the very thing I wanted to do! Just think how much more could be done if instead of independently working on parallel projects we came alongside each other to consolidate efforts. In order for this to happen, we have to be able to know what each other is doing, and this is why we have recently launched the Open Scriptures Directory, which seeks to comprehensively list open projects involving scriptural data.

In addition to the Directory, the Google Group is also serving as a way for people to collaborate. This week there has been an active thread discussing collaboration on Strong’s Dictionary data. David Troidl expressed the general sentiments well:

The idea of collaboration is great. I’ve already mentioned to Darrell that just the moral support is heartening. It’s nice to know, after all this time working alone, that there really are others out there who share the same interest. […] I’ve been on my own all this time

I think it is essential to seek out others who are interested in the same things so that we can come together not only to build upon what each other has done, but to consolidate what we are doing as much as possible so that both duplicate efforts and fragmented data can be avoided. Not only will this lead to more productivity, but it will also serve to build community among those who share a common vision for open scriptural resources.

If you have a project that is not yet listed in the directory, please suggest the link. Please also join the Google Group!

BibleTech:2009 was a success! The conference was a great opportunity to learn about the cutting edge developments at the intersection of Bible and technology, and to meet the people behind them. I’ll summarize some of the connections that relate to Open Scriptures.

I finally got to meet the people behind the Tagged Tanakh project from the Jewish Publication Society; they presented in the talk “How the Ancient Rabbis Invented Web 2.0 Before Its Time.” When I learned about their project earlier this year, I got really excited because the Tagged Tanakh’s vision is almost identical to that of Open Scriptures except for its focus on the Jewish scriptures. JT Waldman is directing the project, and we really connected both personally and professionally. We immediately began collaborating and strategizing on how we can assist each other and work together to realize the common vision.

Bible.org was represented at the conference, and they also want to work together. Bible.org has had a very open policy with regard to licensing their New English Translation (NET), and they are looking for new ways to make it even more openly accessible. Open Scriptures is one such avenue, in addition to their existing NET Bible study tool. The NET Bible is a solid translation that includes a multitude of scholarly notes which will profoundly benefit resources that are integrated into the scriptural Semantic Web of Linked Data (again, see Tim Berners-Lee’s TED Talk).

Stephen Smith presented on “The Need for a Universal Bible Annotation Format.” Stephen, previously employed by Crossway as the developer of the ESV Online, is now employed at Zondervan and is working to bring the same openness he built into ESV Online to be taken to a larger scale at Bible Gateway, currently the most popular Bible website. Stephen is also the voice behind OpenBible.info where he, obviously, promotes openness of scriptural data. In his talk Stephen described a data format that would enable not only the portability of users’ data around the Web and among their devices, but it would also allow web apps to integrate scriptural data from across the Web, making possible new applications powered by this data. His talk examines the requirements for such a system and it outlines possible ways to implement it. He announced that the Open Scriptures Google Group would be where collaboration on such standardization would take place (see threads one and two).

Sean Boisen of Semantic Bible, who last year presented on Bibleref, this year presented on the Bible Knowledgebase (BK). The project’s goal is to identify and mine all of the people, places, and things in the Bible and to connect them all together into Linked Data. While this is a proprietary product of Logos, there is hope that the identifiers (URIs) used will be published so that the community can standardize on a common namespace.

And lastly, of course, I presented the Open Scriptures project itself, and I am thankful for how well it turned out. The multimedia from the talk is available.

I just ran across a relevant and extremely important TED talk by Tim Berners-Lee, inventor of the Internet, regarding “The next Web of open, linked data,” in which he presents the case for Linked Data. Open Scriptures is a Linked Data initiative, seeking to integrate and semantically interlink all of the scriptural data available. Berners-Lee notes in his talk:

The really important thing about data is the more things you have to connect together, the more powerful it is.

This is so key. If you have raw data for two resources, there is only a limited number of ways that this data can be combined together. However, each time you add in another raw data set, the number of combinations grows exponentially. Check out his talk…

Again, Open Scriptures is foundationally about Linked Data for Scripture, granularly interconnecting scriptural texts at the semantic level, and making this data openly accessible. The more links added between texts will result in views and applications of scriptural data never before possible!

Raw Data Now!

(The following project writeup I did back in October of last year, but it has not been published until now.)

At the BibleTech 2008 conference, James Tauber of MorphGNT identified the need for the wide array of scriptural data to share common references so that they could be integrated and mashed up. However, even if the data we have today shared common references, the ability to integrate this data would be out of reach for the general public. The Composite Gospel Index (CGI) at Semantic Bible, for example, identifies parallel pericopae in the gospels; it makes available XML data consisting of OSIS identifiers which identify and group parallel pericopae. The project includes a view of the data with the text of the RSV. If, however, an Arabic speaker wanted to view the parallel pericopae in an Arabic translation, they would be unable unless they did it by hand or had experience writing applications which parse XML and query a Bible web service (if one even exists for Arabic translations); moreover, if someone else desired to view the CGI in another translation, then the same work done with the Arabic translation would have to be done all over again. Open Scriptures seeks to provide both the unified data repository for serving scripture (such as a NT manuscript or an Arabic translation), the internationalized or language-independent derived scriptural data (such as the CGI), the API to query the data repository, and a hosted application framework which allows work done once for one translation to be immediately available for any other translation.

Open Scriptures is a repository for Biblical manuscripts and their translations, and a system for storing the differences between manuscripts and their relationships to versions expressed by semantic links: it seeks to represent the textual transmission of the Bible and, on top of this foundation:

1. supply an Open interface for querying interlinked scriptural data,
2. store derived data in an internationalized (i18n) and translation-neutral manner, and
   
3. provide an application platform for mashing this data up into scriptural applications with a framework for discussion and collaboration.

The most fundamental application of Open Scriptures is the comparison between one manuscript and another, between manuscripts (MSS) and their translations, and between the translations themselves. This base application provides new ways to do textual criticism so that in addition to seeing the differences that exist, but to also discover and discuss why there are differences between MSS and translations. This is possible because the MSS are merged into a kind of “unified diff” and because the semantic units in each translation are linked to the semantic units in this unified MS diff from which they were translated. (These semantic-unit links are contributed by users who desire to have a translation added to the system.) At BibleTech 2008, Karl Hofmann set forth (MP3) a vision for just such a need in Bible software, to provide:

Not only what are the differences between texts, but why are there differences? We have to go back and find out the decisions that were made along the way. […] Software must allow us to discuss the text, not just concepts.
[…]
How is it that we are going to be able to get these tools to come together and provide the ability to make distinctions and show where the disagreements lie at a textual level as opposed to at a conceptual level. So instead of talking from my presupposition to your presupposition, we’re actually going to be able to talk about the text.
[…]
A tool that allows us to recognize the distinctions that are already being made, the judgments that are already there, and instead of trying to change your mind from the judgment you’ve made to the judgment I’ve made, to go back and find out from what point, at what point was that judgment made. Where was the distinction made, on what basis was it made? Was it made because I have a presupposition about how the text was read at a particular time? […] or is it made because of some semantic misunderstanding I have? All of the different ways that we can be judging can be brought to light and examined and at least I can make an informed decision…

Thus Open Scriptures seeks to present the texts of the Bible as a social media, and that the functionality provided will inspire collaborative interaction at the textual level.

A second fundamental benefit of having a unified manuscript and having each translation independently link to it is that any scriptural project undertaken using the text from one translation will automatically be available to any other translation that also has its semantic units source-linked. The plethora of Bible translations is causing a fragmentation and isolation of scriptural projects and resources. Countless word studies on any particular word have been independently undertaken, each time perhaps using a different translation or language. If such a word study were to be done on the Open Scriptures framework, multiple users may collaborate on a word study project using their own preferred translation and the results could be viewed in any translation; projects such as cross-references would also especially benefit from this. Scriptural projects become social media developed on a social web.

The applications that are possible on the Open Scriptures framework are exciting; it makes trivial the creation of bilingual editions, interlinear views, full-text searching and exhaustive concordances of words from multiple translations in a language, version-independent word studies and cross references with internationalized expositions.

Open Scriptures is part of the open source movement and will operate not-for-profit; it seeks to make the Scriptures available to the most number of people and provide them with the tools they need to study them and to freely share their findings. Open Scriptures will be built utilizing open web standards and best practices in web development to create an accessible RESTful web service. The state of the Web is such that all of the pieces are coming into place:

• Unicode is now ubiquitous and so the text of the Biblical languages can be easily processed and delivered.
• The Open Scriptural Information Standard (OSIS) provides the basic XML vocabulary necessary for encoding scriptural texts.
• New efforts to photograph and transcribe Biblical manuscripts in electronic format is providing the necessary foundational textual data (see Biblical Manuscripts Project, CSNTM, and The Codex Sinaiticus Project), aside from the numerous manuscripts and resources that are already freely available, such as MorphGNT.
• Web browsers are powerful applications which can handle complex web applications employing state-of-the-art technologies like Ajax, SVG, Canvas, and HTML 5.
• Cloud computing (as in Google AppEngine, Aptana Cloud, or Amazon EC2) is maturing and provides an architecture to power high-demand processing of complex data structures.
• Social media and the Social Web are instilling certain expectations with regard to how we interact on Web 2.0 (as with blogs, Facebook, and Wikipedia).
• Google has raised expectations on the capabilities of web applications, their openness, and how data on the web can be mashed up.
• Google and Wikipedia have accustomed us to having a central resource for finding information by bringing together diverse fragmented data into an integrated whole.
• The Internet is becoming increasingly global and people from every tribe, tongue, and nation are coming onto the Web; the nations need an internationalized service for studying the Scriptures.
• Mobile web devices are becoming increasingly popular due to the iPhone, so a web service accessible to mobile devices can satisfy the needs of a rapidly increasing number of mobile users.

Open Scriptures seeks to integrate all of these pieces. For thousands of years scribes copiously copied manuscripts by hand onto vellum or papyrus with a pen and ink. They took great care to ensure that the text was accurately and reverently transferred and made their manuscripts beautifully ornate works of art; they glorified God with the work of their hands. Now today, instead of pen and paper, we have Unicode and HTML; instead of scribes, we have software developers; instead of codices, we have websites. Open Scriptures seeks to apply the same level of skilled craftsmanship in web development as the scribes’ own skilled craftsmanship in the presentation of the Scriptures, all to the glory of God and the edification of His people.

As Zack Hubert said at the conference last year, “It’s a community effort. Any time anything good happens, is because a real cool team of people have come together around an idea.” Open Scriptures seeks to be such a community effort.

Update 3: Looking for a website that provides the same core functionality of Zhubert.com? Check out John Dyer’s excellent Reader’s Version of Greek and Hebrew Bible. It’s free!

Update 2: See post on the group regarding how to respond to copyright issues.

Update 1: zhubert.com has been shut down due to the discovery that the MorphGNT has infringed on the German Bible Society‘s copyright of the UBS4 edition of the Greek New Testament. Please do not flame zhubert.com or the German Bible Society or anyone else in the comments here. Each is doing what they feel is right given the situation. We are working on a solution to the copyright issue.

Zack Hubert (former Pastor of Technology at Mars Hill Chruch, now Vice President of Church Community Networks at Zondervan after they aquired The City) spoke (MP3) last year at BibleTech about the history of his free NT Greek web app zhubert.com. After years of development he came to a point in life where he had no time to maintain the project, let alone take it to another level. He shared his desire to further the project and his first plan to do so:

(22:32) What can I do to keep this project continuing since I only had 15 minutes a week to go in and either approve a lexicon entry or make a little bug fix. So I came up with a plan. The plan was the Biblios foundation. (I say past-tense for good reason.) So I started off down the road of creating a non-profit organization. And the idea was that it could accept donations and that I could have this self-supporting organization where I’d draw no income but I could use it to contract out developers… you know find other guys that were right out of college or in seminary or what not and give them $15 an hour or some ridiculously small amount to add on the features and functionality that I think really would have taken zhubert to another level… but I’m not a business man, and yeah… it failed. Quite simply, looking back on it I think people just wanted the software for free. […] (24:25) So development of that just took a break. I was kinda disappointed honestly because I really wanted to see this project go, but there seemed like very few people who were getting behind to help me, so I took a mental break from the project.

He then shared his next plan, to transition his privately-developed closed-source project into an open-source community-driven one called the “Resurgence Greek Project” or simply “Re:Greek”. In an interview (via Wayback Machine since original post inaccessible) he shared:

My vision for Re:Greek is that it would be an Open Source project that could capture the imagination of software developers all around the world. I want to see dozens of developers and designers submitting improvements, new features, localizing Re:Greek into their local language, and innovating on this shared platform. I’ll release more in the near future about how this will work, but it will be a model similar to Linux or Ruby on Rails if you are familiar with those projects.

In another post (also via Wayback Machine since original inaccessible) he was “happy to announce that we have made Re:Greek an Open Source Initiative!” Unfortunately, as you’ll learn if you listen to his BibleTech talk (see above), the initial excitement was extinguished and the project transition failed. People who were used to the old zhubert.com were upset with the new Re:Greek project and just wanted the old site back. There weren’t enough developers and supporters to sustain the Re:Greek open-source project, and so it ceased. I don’t know all of the details, but the project was so thoroughly discontinued that Zack’s announcement blog posts have been removed and the ReGreek Google Group has been deleted. Zack seems to have washed his hands of the Re:Greek project and has thereafter focused all his efforts on The City (which is turning out to be a great success). Such a sad turn of events for such a brilliant idea!

At BibleTech this year, I am presenting the Open Scriptures project as “Picking Up the Mantle of the Re:Greek – Open Source Initiative”. We need to give Zack’s idea a second chance, but we need to learn from his experience. Open Scriptures will also fail if it does not have the collaboration and co-ownership of committed developers and the sponsorship of supporting organizations. However, if the Mozilla Foundation can make Firefox such a success, if the Linux community can persevere to make a solid open-source OS, and if Wikipedia is able to become the global first source for encyclopedia information, then surely there are people of faith—who believe their scriptures are the very words of God—who can make Open Scriptures a reality.

Will you join me?

Join the conversation at the Open Scriptures Google Group and spread the word.