{"id":387,"date":"2019-04-01T08:54:10","date_gmt":"2019-04-01T13:54:10","guid":{"rendered":"https:\/\/cdrhdev.unl.edu\/log\/?p=387"},"modified":"2019-04-01T08:54:10","modified_gmt":"2019-04-01T13:54:10","slug":"land-of-confusion-a-relationship-visualization-experiment","status":"publish","type":"post","link":"https:\/\/cdrhdev.unl.edu\/log\/2019\/land-of-confusion-a-relationship-visualization-experiment\/","title":{"rendered":"Land of Confusion: A Relationship Visualization Experiment"},"content":{"rendered":"

In September 2015, we launched the O Say Can You See: Early Washington D.C., Law & Family<\/em> website (OSCYS) [http:\/\/earlywashingtondc.org<\/a>].\u00a0 The site had an interactive network component, where users could easily see how a petitioner not only had relationships to attorneys and defendants, but to a web of family members and social connections involved in other court cases.\u00a0 What we didn’t know then is that the networks would quickly grow into incredibly tangled webs that would need to be revisited in 2019.<\/p>\n

\"O
O Say Can You See’s logo emphasizes the project’s network of relationships<\/figcaption><\/figure>\n

In 2015, before the visualization tool could be born, Laura Weakly and Kaci Nash had already spent hours upon hours recording relationships between all the individuals they could identify in court documents, warrants, minute books, petitions, and more.\u00a0 As the team’s developer, I put myself through a crash course in RDF and SPARQL queries as I worked on figuring out how to present the relationships in a way that could be interrogated, and then I experimented with JavaScript visualization libraries that would allow the results to be explored by users.\u00a0 I settled on the JavaScript InfoVis Toolkit (JIT) [http:\/\/philogb.github.io\/jit\/<\/a>], inspired by the Social Networks and Archival Context (SNAC) [http:\/\/snaccooperative.org\/<\/a>] website’s use of the library.\u00a0 As I recall, I struggled for a little while getting JIT to color lines correctly for the relationships between individuals, but ultimately we ended up with something we were all happy with, we launched the site, and I moved onto the next project.<\/p>\n

Laura and Kaci continued adding new documents, however, and that meant adding more people.\u00a0 Soon some of the visualizations were starting to break browser tabs — “Warning: unresponsive script” messages were showing up as the user’s browser scrambled about, frantically trying to draw thousands of points and lines.\u00a0 We restricted judges and clerks from showing up in the visualizations, thinking that since those people were involved in almost all of the court cases and didn’t necessarily have personal relationships with many of the individuals we were most interested in, perhaps this would help the visualization.\u00a0 It continued to limp along, but active attorneys like Francis Scott Key continued to grow into larger and larger monster networks.<\/p>\n

Finally this spring, we were able to find time to revisit the project.\u00a0 Although I was supposed to be primarily building a few tools for the project team’s workflow and working on a pretty sweet map,\u00a0I spent some spare time trying to mitigate some of the worst concerns about the visualizations.\u00a0 Speaking of that pretty sweet map, it’s live now!\u00a0 Take a look at the work Kaci Nash and Patrick Hoehne did putting together a map of Washington, D.C., 1822 directory data, and the people in OSCYS documents here: [http:\/\/earlywashingtondc.org\/maps\/directory_dc<\/a>],<\/p>\n

\"historical
You can explore the map with layers like court role, occupation, race, and gender<\/figcaption><\/figure>\n

Back to the visualization:\u00a0 I tried out d3 [https:\/\/d3js.org\/<\/a>] first, having worked with it once before for another of Dr. William Thomas III’s projects, Reconstructing African American Mobility After Emancipation, 1865-1867 <\/em>[http:\/\/railroads.unl.edu\/mobility-after-emancipation\/<\/a>].\u00a0 I have found that d3 is very powerful, but also very confusing to quickly pick up and customize for a casual user. I played around for a bit with radial dendrograms and tidy trees, but foresaw a long and mathematical struggle coming when it came to trying to prevent labels from overlapping colorful lines, among other potential hurdles.\u00a0 Another obstacle I encountered is that the radial dendrograms and trees don’t “share” relationships — people in the second-removed set of individuals would be duplicated multiple times for each first-removed individual who knew them.\u00a0 In the current OSCYS visualization, these individuals are shared and the network appears much more “web” like compared to the orderly d3 model.\u00a0 At least for now, without a lot more time to dedicate to learning the dark arts of d3, I decided to keep looking.<\/p>\n

\"circle
An example of a d3 tree from https:\/\/observablehq.com\/@d3\/radial-tidy-tree<\/figcaption><\/figure>\n

I remembered playing around with a library called vis.js [http:\/\/visjs.org\/index.html<\/a>] back in 2015 and that I had enjoyed the straightforward documentation and fun “physics” behavior of the points and lines.\u00a0 Unfortunately, the original project is no longer being maintained, but there are enough friendly forks of it that I decided to experiment a little, anyway.\u00a0 If it turned out to be able to handle tons and tons of data points, then one of those community maintained forks might look pretty attractive.\u00a0 I hooked up the SPARQL query to send JSON that the vis.js library could understand and took a look.\u00a0 For people with relatively few connections, it looked beautiful.\u00a0 Changing the points, lines, and labels was easy, it might not be so very labor intensive to modify the OSCYS network code which highlights types of relationships, pan, zoom, and more to use this library.<\/p>\n

\"chart
So far so good! A vis.js relationship network with bright colors that I created from real data while testing things out<\/figcaption><\/figure>\n

 <\/p>\n

\"cluster
A look at a zoomed in cluster while testing vis.js<\/figcaption><\/figure>\n

Yet, vis.js’s greatest challenge remained…….a visualization of somebody like Francis Scott Key!\u00a0 With some trepidation, I put in the URL and hit enter.\u00a0 What followed, I feel, is best represented in images rather than text.<\/p>\n

\"Francis
Oh no<\/figcaption><\/figure>\n

 <\/p>\n

\"Dandelion
The physics of the points and lines is starting to shake things out<\/figcaption><\/figure>\n
\"A
Francis Scott Key’s relationships make a very pretty flower, I daresay<\/figcaption><\/figure>\n

 <\/p>\n

Interestingly, though vis.js was also struggling with that much data, it did display everything for Francis Scott Key (although with an awful lot of spasms and twitches from the physics driving the point location), while JIT wouldn’t necessarily get that far without an awful lot of patience by the user.\u00a0 Finally being able to visualize all of Key’s relationships, however, did make it very clear that at some point we needed to ask ourselves what the visualization was for.\u00a0 Key knew too many people to navigate that type of interactive visual and come away with any particular understanding besides “wow he knew a lot of people through being an attorney.”<\/p>\n

So what DID we want to communicate? We wanted to be able to show if a person had many connections or relatively few (keeping in mind these are only to people on the OSCYS site, therefore somehow connected to a court case we’ve documented).\u00a0 We wanted to be able to show the type of relationship (family, legal, social, etc) in such a way that at a glance you can see the proportions of such relationships.<\/p>\n

Our imperfect solution was to restrict how many results are coming back and make note of the restriction in the visualization.\u00a0 If a person has several relationships, and one of those related people has dozens and dozens of connections, we are not displaying that particular related person’s secondary connections.\u00a0 If a person has scads of immediate connections, then we do not display the second ring of connections with a relationship to all those immediate connections.\u00a0 This strategy means that the visualization charts are once again usable.\u00a0 You can still get an idea of how well connected an individual was in the OSCYS world with the numbers of omitted individuals.\u00a0 You can still explore the types of immediate connections.<\/p>\n

\"Huge
Francis Scott key’s visualization is still a little bit silly even with the secondary relationships omitted<\/figcaption><\/figure>\n

Though the secondary connections may be less accessible, now, it seems less critical to know that a person’s attorney knew many people (which the original person may not have interacted with) through the courts, than that this person had an attorney, a brother, a neighbor, and a slaveholder against whom they were petitioning.<\/p>\n

<\/div>\n

\"graphNetwork visualization with several individuals’ connections omitted<\/p>\n

An unexpected perk of handling the visualization problem through limiting hugely well-connected individuals is that we were able to add clerks and judges back into the mix and so note the number of their relationships, whereas before they were entirely absent from the chart.<\/p>\n

I’m not entirely happy with this solution, of course.\u00a0 I would rather find some way to present the networks which is straightforward and usable, but at least for now, the interactive network visualizations have survived when it appeared that this portion of the site might need to be shuttered.\u00a0 Perhaps in the future we will be able to revisit them and find either a new way of presenting the information or come across a new method for presenting them in a similar but more manageable way!<\/p>\n","protected":false},"excerpt":{"rendered":"

In September 2015, we launched the O Say Can You See: Early Washington D.C., Law & Family website (OSCYS) [http:\/\/earlywashingtondc.org].\u00a0 The site had an interactive network component, where users could easily see how a petitioner not only had relationships to attorneys and defendants, but to a web of family members and social connections involved in…<\/p>\n Read More<\/span> Land of Confusion: A Relationship Visualization Experiment<\/span><\/a>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"Layout":"","footnotes":""},"categories":[9,6,37,22],"tags":[54,52,51,11,13,53],"class_list":["entry","author-jdussault","post-387","post","type-post","status-publish","format-standard","category-linked-data","category-network","category-research","category-usability","tag-d3","tag-jit","tag-network","tag-oscys","tag-sparql","tag-visjs"],"_links":{"self":[{"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/posts\/387","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/comments?post=387"}],"version-history":[{"count":7,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/posts\/387\/revisions"}],"predecessor-version":[{"id":407,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/posts\/387\/revisions\/407"}],"wp:attachment":[{"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/media?parent=387"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/categories?post=387"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cdrhdev.unl.edu\/log\/wp-json\/wp\/v2\/tags?post=387"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}