{"id":225,"date":"2017-04-21T10:24:15","date_gmt":"2017-04-21T15:24:15","guid":{"rendered":"https:\/\/cdrhdev.unl.edu\/log\/?p=225"},"modified":"2017-04-21T11:03:59","modified_gmt":"2017-04-21T16:03:59","slug":"day-of-dh-1","status":"publish","type":"post","link":"https:\/\/cdrhdev.unl.edu\/log\/2017\/day-of-dh-1\/","title":{"rendered":"Day of DH: Data Data Data"},"content":{"rendered":"

[This post is also available on the Day of DH’s blog<\/a>]<\/p>\n

Hello! \u00a0My name is Jessica Dussault and I am a programmer at the Center for Digital Research in the Humanities<\/a>\u00a0(CDRH) at the University of Nebraska-Lincoln<\/a>\u00a0(UNL). \u00a0Let me tell you a little about my Day of DH 2017 which was, if you can tell from the title, filled with data.<\/p>\n

Hawking Lewis and Clark<\/h2>\n

My morning started off with conversation and some brainstorming. \u00a0In late 2016, we redid almost everything about the\u00a0Journals of the Lewis and Clark Expedition<\/a> website,\u00a0updating the TEI XML<\/a>\u00a0and\u00a0redesigning the navigation, appearance, and technology<\/a>. \u00a0As one of our sites with the highest traffic and an important educational resource, it was important to us to make sure it continued to be accessible to all users in the future. \u00a0Now, the work behind us, we find ourselves in the \u201cmarketing\u201d phase of the project. \u00a0How do we let people know about the site\u2019s redesign and new features? \u00a0Should we queue up tweets for each day\u2019s\u00a0journal entries and tag the Lewis and Clark National Historic Trail<\/a>? \u00a0Perhaps post a recipe for\u00a0boudin blanc<\/a>\u00a0<\/em>to social media? \u00a0Create a choose-your-own-adventure Lewis and Clark game in the style of\u00a0the Field Museum\u2019s account<\/a>? \u00a0After an informal discussion in the morning, we came away with plenty of ideas\u2026.stay tuned to see how it turns out!<\/p>\n

Independent\u00a0Research Time<\/h2>\n

The CDRH has been kind enough to put 10% of the dev team time towards independent research. \u00a0This could be entirely tech related, DH related, or, in my case, local history related. \u00a0As an undergraduate at UNL, I was a member of the Cornhusker Marching Band<\/a> and became interested in its history. \u00a0As a staff member, I\u2019ve been able to pursue that interest\u00a0and recently CDRH and the University Archives and Special Collections<\/a> (SPEC) have digitized three<\/a> of ten vulnerable reels of band footage from the 1940s to 1960s. \u00a0My coworker Sara Roberts and I also just finished teaching an OLLI class on the history of the marching band at UNL, so yesterday I took a bit of time to\u00a0tie up loose ends \u2014 promising to clean up and publish the presentations, corresponding with guest speakers from the class, and tackling a few of the somewhat incomplete Zotero<\/a> references we\u2019ve been collecting. \u00a0It\u2019s so easy\u00a0to click the \u201ccite\u201d button in a browser and move on, rather than immediately filling in more information about the source. \u00a0Sara and I have a lot of dream plans, like locating funding to repair and digitize the remaining reels, writing up something more formal about the band\u2019s history, creating a digital media project, and recreating drill routines from the 20th century. \u00a0There\u2019s a lot to do, but a few hours a week, we\u2019re cranking closer and closer to the next big project.<\/p>\n

One API to Rule Them All<\/h2>\n

Most of my day was spent working on the Center\u2019s API effort. \u00a0Karin Dalziel dreamed up a center-wide API a few years ago, but we\u2019ve only started going all-in on it the past few months. \u00a0Most of our projects are in TEI<\/a> and furthermore, many of them are ingested into Solr<\/a> for project specific searching functionality. \u00a0It makes a lot of sense to be able to search across all of them at the same time, right? \u00a0Unfortunately, in the past ten years as each project was created, the Solr fields used had similarities, but weren\u2019t standardized. \u00a0Here\u2019s the current API plan:<\/p>\n

    \n
  1. Create a set of standard fields and fieldtypes for the API<\/li>\n
  2. Identify the search platform<\/li>\n
  3. Ingest TEI (and other formats) into search platform<\/li>\n
  4. Design API<\/li>\n
  5. Build API<\/li>\n
  6. Build frontend to stand up websites per project quickly<\/li>\n
  7. TEST TEST TEST<\/li>\n
  8. Start using it for real<\/li>\n
  9. Party<\/li>\n
  10. See if other people want to use it, too?<\/li>\n
  11. Speaking tour of the country talking about how awesome it is<\/li>\n<\/ol>\n

    Right now we\u2019re mostly on steps 5 and 6 (step 7 is technically ALL STEPS), but yesterday I doubled back to work on step 3. \u00a0Maybe two years ago I had designed something to take TEI and prep it for Solr, but since choosing Elasticsearch<\/a> I had mostly abandoned the Solr ingest scripts for the new \u201cdata repository\u201d as we\u2019re calling it. \u00a0However, since many of our old sites are still powered with Solr, it\u2019s important to be able to quickly manage those indexes (creating, schema-fying, populating, clearing, etc), so I worked yesterday on plugging the functionality back in. \u00a0I also got to delete some old code that is no longer being used, which was super satisfying. \u00a0Nothing quite so nice as removing code that reminds you of how you once were a terrible \u2014 I mean how much you\u2019ve grown as a \u2014 programmer.<\/p>\n

    This plugging back in and then tearing out took me most of the day. \u00a0I will probably be doing more of it today, as well, with a particular emphasis on writing unit tests!<\/p>\n

    Endangered Data Week<\/h2>\n

    Endangered Data Week<\/a> is in full swing! \u00a0It\u2019s described as \u201ca new, collaborative effort, coordinated across campuses, nonprofits, libraries, citizen science initiatives, and cultural heritage institutions, to shed light on public datasets that are in danger of being deleted, repressed, mishandled, or lost.\u201d \u00a0Yesterday I went to a session held at the UNL Library that introduced the concept of endangered data and looked at some current strategies being pursued. \u00a0We thought up some ideas as a group of what we can do on a more local level. \u00a0Although I think Endangered Data Week is not targeted towards data like the marching band film reels I previously mentioned, I kept thinking about them. \u00a0In some ways, the UNL Library is already trying to \u201csave\u201d some data which is in danger of becoming unplayable. \u00a0At the same time, people raised really great points about how we could be helpful in requesting information from Nebraska state government agencies, identify agencies whose documentation and data may not always be available, and take an overall survey of what information is available online for Nebraska organizations.<\/p>\n

    What\u2019s Next?<\/h2>\n

    More unit tests! \u00a0My Friday probably contains some code review and cleanup of ongoing API components. \u00a0It sounds like I\u2019m also going to be helping put together a Poetry Printer<\/a> with some of my coworkers, which should\u00a0be a fun Friday project. \u00a0I also need to send out a reminder email that the next Dish with DH event is on Monday, which will give folks on campus interested in Digital Humanities a chance to meet up over lunch and chat about what they\u2019re working on. \u00a0I hope you enjoyed this brief cross-section of all the programming and non-programming I do at my job at the CDRH!<\/p>\n","protected":false},"excerpt":{"rendered":"

    [This post is also available on the Day of DH’s blog] Hello! \u00a0My name is Jessica Dussault and I am a programmer at the Center for Digital Research in the Humanities\u00a0(CDRH) at the University of Nebraska-Lincoln\u00a0(UNL). \u00a0Let me tell you a little about my Day of DH 2017 which was, if you can tell from…<\/p>\n Read More<\/span> Day of DH: Data Data Data<\/span><\/a>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"Layout":"","footnotes":""},"categories":[32,37],"tags":[38,36,39,35,34,26,27,24,33],"class_list":["entry","author-jdussault","post-225","post","type-post","status-publish","format-standard","category-api","category-research","tag-band-history","tag-day-of-dh","tag-digitization","tag-elasticsearch","tag-endangered-data","tag-lewis-and-clark","tag-solr","tag-tei","tag-unit-test"],"_links":{"self":[{"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/posts\/225","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/comments?post=225"}],"version-history":[{"count":2,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/posts\/225\/revisions"}],"predecessor-version":[{"id":227,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/posts\/225\/revisions\/227"}],"wp:attachment":[{"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/media?parent=225"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/categories?post=225"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/tags?post=225"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}