Open Scriptures

I have been researching stand-off markup, and discussing (and discussing) on osis-users about its applicability as a normalized way in OSIS to represent scripture in a single consistent XML structure. I was initially introduced to stand-off markup by Efraim Feinstein of the Open Siddur project and then further by James Tauber (1, 2, 3, 4, 5). I just came across the following quote in TEI P5: Non-hierarchical Structures:

It has been noted that stand-off markup has several advantages over embedded annotations. In particular, it is possible to produce annotations of a text even when the source document is read-only. Furthermore, annotation files can be distributed without distributing the source text. Further advantages mentioned in the literature are that discontinuous segments of text can be combined in a single annotation, that independent parallel coders can produce independent annotations, and that different annotation files can contain different layers of information. Lastly, it has also been noted that this approach is elegant.

But there are also several drawbacks. First, new stand-off annotated layers require a separate interpretation, and the layers — although separate — depend on each other. Moreover, although all of the information of the multiple hierarchies is included, the information may be difficult to access using generic methods.

In the current OSIS schema, one structure (e.g. verse, quote, or paragraph) has to be chosen as primary, leaving the others to be represented with milestoned elements (actually, only verses can currently be chosen as primary); stand-off markup would allow overlapping hierarchies of verses, quotes, and paragraphs to all be on equal footing without having to choose one as primary. Is this good for OSIS? Please join in on the discussions at osis-users!

A couple of us are presenting at the BibleTech:2010 conference in San Jose, California. Weston Ruter is presenting:

Open Scriptures API: Unified Web Service for Scriptural Linked Data

The OSIS XML standard provides for a lot of free variation in the way it represents scriptural constructs (such as verse boundaries). Because of this, different OSIS documents encoding the same work may have vastly different DOM trees, which make automated traversal of arbitrary OSIS documents very difficult. Aside from this fact, the DOM is not a very programmer-friendly way to query for scriptural data to begin with. In this era of web services and mashups, having a standard, unified way to access scriptural data is a prerequisite for scriptural applications to take off in the same way that applications based on other common datasets have (such as maps). Furthermore, these scriptural datasets should be all explicitly interconnected as Linked Data of the Semantic Web, so that any metadata attached to a word in one translation would also be available to any other translation or manuscript by means of their interconnections. So while OSIS XML is “a common format for many visions”, this talk will explore “a common API for many datasets”; this will be a continuation of BibleTech:2009’s talk: “Open Scriptures: Picking Up the Mantle of the Re:Greek – Open Source Initiative”.

James Tauber is presenting, in addition to “A New Kind of Graded Reader”:

Using Pinax and Django For Collaborative Corpus Linguistics

Django is a popular, Python-based Web framework. Pinax is a platform for rapidly building sites on top of Django, particular sites with a strong collaborative focus.

After introducing Django and Pinax, we will discuss Pinax-based tools the speaker is developing to help with web-based collaboration on corpus annotation with applications from lexicography to morphology to syntax to discourse analysis.

And there are many other fascinating talks lined up. Hope to see you there!

Recently, there has been a surge of posts on Open Scriptures. If you haven’t been able to follow them all, here are some of the most exciting threads.

• The Open Scriptures code is moving to GitHub, from Google Code Project Hosting. See our page http://github.com/openscriptures.
• The Open Scriptures website is being redeveloped in Pinax. This is a major change which will allow us to define our projects more clearly as well as become the platform upon which we can build our applications. We are looking to abandon Google Groups. See the Openscriptures.org Redevelopment thread for more information. Special thanks to James Tauber at Eldarion for his generosity in hosting and setting up our new site! We’ll be announcing the beta site soon.
• New group activity kicked off by post, Standardizing on a Web Infrastructure and Web Service API for Scripture, sent to Bible Technologies Group, Crossway, CrossWire, Bible.org, and others. We discussed work on a new edition of OSIS. We are actively talking with Bible.org (NET Bible) and the Crossway (ESV) about ways to promote openness and interoperability.
• We’ve started some excellent dialogue with the Open Siddur project and their lead developer Efraim Feinstein.
• Reminder about preserving the right for a worker to receive his wages.
• The group decided on a common license for our projects, Creative Commons BY-SA. The bulk of the discussion may be found under WLC Lexical Tagging.
• We discussed our channels of communication in the Regular Meetings over IRC or Google Wave thread. The unofficial conclusion is that we do need to have some regular real-time meetings. However, the new Pinax site will also allow for focused collaboration that may alleviate some of the barriers we are experiencing.
• We also decided to use the name “MorphHB” for an open project collecting the morphology of the Hebrew Bible. The thread Decision about how to name the project – Morphology and the WLC has more information.
• We discussed word-level linking between texts.
• We are also now nailing down our the goals for the project, and will be releasing a more refined project description soon.
• We also discussed the naming of our project. The “slug” we used on Google was “open-scriptures” but we are standardizing on “openscriptures”, as on Twitter and GitHub.

There are many more important threads that I will leave for your perusal at our Google Groups site. Over the course of the next few weeks expect to see some more explanations of what we have been discussing. Please feel free to join in the conversations!

The exciting news this week is the dialogue that Weston has been carrying on with Dr. Bertram Salzmann from the German Bible Society. In a nutshell, we are working together to create a developer platform that will give access to the copyrighted texts that GBS maintains (such as the renown UBS GNT) along with other openly-licensed works already available online. The conceptual outline that Dr. Salzmann has proposed keeps GBS’s texts under their umbrella by means of hosting the texts and the applications that make use of them. This is somewhat different than the original idea that Weston proposed in which Open Scriptures would be more of a true mediator between open source developers and content providers like GBS. In any case, the applications would be made available free of charge. The exact details have yet to be figured out. Many thanks to Dr. Salzmann and GBS for their innovative forward-thinking proposal! Please help by joining in on the conversation!

The past few weeks on Open Scriptures have seen some steady progress.  Of note is that the Tregelles GNT import script is near completion.  The addition of this manuscript will be much appreciated as it will bring the total number of works in the Manuscript Comparator to six. In addition to New Testament improvements, David Troidl submitted the initial upload of Strong’s Hebrew data (XML).  This first step is the outcome of very hard work and also good collaboration on our Google group. Lastly, we are working on porting all of our code into a Django/Pinax friendly format so that we can switch our site as well as our applications over to this platform.  The progress here is moving along and within a couple weeks we should be on our new server using our new platform graciously donated by James Tauber of MorphGNT and Eldarion.  If anyone has experience with Django/Pinax and would be willing to help out please contact Weston via the Google Group.

There has also been some significant steps forward in the dialog between Weston and the German Bible Society; GBS has a tentative proposal which looks to be quite promising and beneficial for both communities. Look forward to an announcement hopefully next week.