Where is Web Science? From 404 to 200

What I cannot create I do not understand. — Richard Feynman

There are several examples of well-intentioned researchers presenting their findings and ideas on how to take scholarly communication as well as Web Science forward, yet the majority stop at actually demonstrating what they propose. To take one recent example, the article The FAIR Guiding Principles for scientific data management and stewardship with 53 authors (that is not a typo), hosted by nature.com (2016), points out a helpful guideline: F1. (meta)data are assigned a globally unique and persistent identifier. To eat one’s own dogfood, we would have expected an identifier for that statement, however it does not exist. Naturally, one should ask: what is Findable, Accessible, Interoperable, Reusable (FAIR) about that statement? What is the measurable delta between this article and every other article (and there are many) making the exact same recommendation? If we were to try to reuse the guideline that is put forward in our study, how can we link to it without any ambiguity? Here is a working example from 1996 https://www.w3.org/DesignIssues/Axioms.html#Universality2 written by Tim Berners-Lee which states: Axiom 0a: Universality 2. Any resource of significance should be given a URI. Why was not this earlier work reused or cited?

In a similar vein, conferences announce their calls for contributions, and require contributions to use desktop/paper-based electronic formats, e.g., ELPUB, ACM Hypertext, WWW, WebSci, ICWE. All of these calls are about sharing research output about the Web as two dimensional objects destined for print. None of them explore the full potential of native Web tools or technologies for knowledge dissemination, or encourage their research communities to do so. Reviews are held behind closed doors, reviewers are not publicly attributed or subject to question. These type of events even state that contributions which do not comply with certain constraints will be rejected at the door without review. To pick further on WWW (the World Wide Web conference), its contribution to the Festival of the Web, expects contributions to be in English, font size no smaller than 9pt, in PDF, formatted for US Letter size, occupy no more than eight pages. McLuhan, Ted, TimBL, sorry, we really messed up our information superhighway. If we apply Tim’s Linked Open Data star scheme, it is debatable whether they would even receive a single ★ (star).

Lastly, the peer-reviews themselves tend to live behind centralised services like Easychair with no particular educational value to anyone outside of the restricted access, i.e., the public. A transparent peer-review process as well as the discussions around it can be seen as an indicator of peer-review quality.

All of these are lost opportunities for the community as a whole, where even small amounts of mindfulness and care on building towards a Web We Want can make a big difference. To counter our complaints, we now present the following as some positive examples of eating one’s own dogfood.

Sparqlines, SemStats, CEUR-WS

SemStats is a workshop series usually held at ISWC. Determined to push forward the vision of accessible and open semantics research, we have been welcoming HTML/RDF based contributions since 2013. The entire workshop site, including the calls themselves are in HTML+RDFa. This enables the possibility to make relations such as article is in reply to a call, as well as having the proceedings semantically related to both the article and the workshop. We can see these simple yet useful links among these documents with their accompanying data:

@prefix sioc: <http://rdfs.org/sioc/ns#> .
@prefix schema: <https://schema.org/> .
@prefix bibo: <http://purl.org/ontology/bibo/> .
<http://semstats.org/2013/>
  as:inReplyTo <http://iswc2013.semanticweb.org/content/call-workshops.html> ;
  schema:hasPart <http://semstats.org/2013/call-for-papers> .
<https://csarven.ca/linked-statistical-data-analysis>
  as:inReplyTo <http://semstats.org/2013/call-for-papers> ;
  bibo:citedBy <http://ceur-ws.org/Vol-1549/> .
<http://ceur-ws.org/Vol-1549/>
  schema:hasPart <http://ceur-ws.org/Vol-1549/#article-06> .
<http://ceur-ws.org/Vol-1549/#article-06>
  bibo:uri <https://csarven.ca/linked-statistical-data-analysis> .

Semantic Web journal

The online Semantic Web journal places a mandate for the authors to provide their peer-reviewed articles in LaTeX format in order to be fit for the third-party publisher’s workflow, as well as to disseminate the research knowledge about Semantic Web in PDF. However this is in addition to the article being available (free of charge) on their website and in some cases linked to their canonical URL. The highlight of the journal is its encouragement for open and transparent peer-review. Reviews are publicly accessible on the contributions at each phase of the publication process. Additionally, authors of the reviews have the option to be transparent about their identity, which plays an important role towards accountability as well as receiving well-deserved attribution for their time and expertise.

Semantic Web Dog Food and ScholarlyData

The Semantic Web Dog Food initiative was aimed at providing information on papers that were presented, people who attended, and other things that have to do with the main conferences and workshops in the area of Semantic Web research. The initiative was successful and grounded itself in demonstrating convincing arguments towards contributing to the Semantic Web vision by providing a recipe for dogfooding. However, one limitation of its corpus on research articles was that it only consisted of metadata like article title, authors, abstract. There was no granular semantic information like hypothesis, arguments, results, methodology, and so on since the article source format was not accompanied with semantic relations.

Following this effort, the ScholarlyData project recently became the successor of Semantic Web Dog Food project, where it focused on refactoring the original dataset, as well as making improvements on the Semantic Web Conference Ontology. Nevertheless, the input data (from the articles, conferences, people and organisations) is still no better than its source formats. The dependency here is not on native Web formats but on others’ requirements and convenience. These initiatives can still have higher potential if the original data shifted from metadata to data.

Current initiatives

Some of the academic conferences in Web Science, e.g., ISWC, ESWC, EKAW, as well as multiple conferences and journals from non Computer Science disciplines, have shown interest in experimenting with the ideas on using non-print centric formats for the research contributions. While the observable progress is relatively slow, it is on the community’s radar. This is a social progress in contrast to the state of affairs circa 2012.

Most recently, venues at WWW2017: Linked Data on the Web pitched the initiative towards Pioneering the Linked Open Research Cloud; and the Workshop on Web Observatories, Social Machines and Decentralisation takes a more holistic approach that is closely aligned with Web Science. The workshop on Enabling Decentralised Scholarly Communication at ESWC2017 is part of an ongoing Call for Linked Research. Contributors to these calls are explicitly requested to make the best use of what the Web offers (incorporating self-publication, archiving, interactions, social engagement), as well as to raise questions to further understand the challenges ahead and ways to pursue them. This ought to be the norm in Web Science scholarly communication regardless of the content of the research itself.

In summary, Web Scientists ought to pay it forward by experimenting with the technologies in their playground when they have the chance. In this day and age, a Web scientist who refuses, neglects, or is incapable of using native Web technologies and standards to capture and disseminate their own knowledge is a bit like a natural scientist insisting to use their own eyes when a microscope is made available to them. We should make best use of the media that are available to us.

Let’s have a look at what it means to exist in terms of scholarly communication.

While DOIs and ORCIDs are useful tools, they are centralised manifestations in context of the Web. On the technical end, information that is identified with a DOI scheme does not in and of itself equate to good quality. After all, anyone can have a DOI registry under their own domain. However in social terms, especially in scientific circles, they are perceived to have more weight than an HTTP URI. HTTP URIs and DOI strings are the same in being unique strings which can be recognised by certain systems. doi.org makes these identifiers resolveable via HTTP, but is extremely brittle, relies on redirection, and suffers from reference rot. The cultural value of DOIs can be observed in the tools and services which are built around only acknowledging or exchanging data which has a DOI. Services like CrossRef or DataCite for instance acknowledge https://doi.org/10.1007/978-3-319-25639-9_5 but do not acknowledge the canonical URI https://csarven.ca/this-paper-is-a-demo, which is accessible to anyone for free, accompanied with granular Linked Data, interactions, peer-reviews, and wider social engagement tooling. Regardless, the former URI is considered to be citable and the latter is just a blog post according to most Web scientists (for those who do not feel like clicking the links, the article contents are the same). Even services like Open Citations (dependent on the other centralised services) only consider resources which essentially have have a DOI pattern.

Similarly, while I can more authoritatively provide rich semantic information about my own identity on the Web through https://csarven.ca/#i (and enabling a follow-your-nose type of exploration) than I ever could with http://orcid.org/0000-0002-0880-9125, the latter gets acknowledged by services which are essentially disjoint from the rest of the Web. Again, current centralised systems would acknowledge the ORCID URI but not what I have to say about myself that is under my control. I own/rent the domain csarven.ca, but have no control over those of orcid.org. Moreover, to ‘park’ a profile at the latter, I must agree to their terms of service. This is not particularly different than having a profile at Facebook, Twitter, or Google for instance. Heir Today, Gone Tomorrow.

Technical reasons are secondary to the issue of social governance here. We should naturally raise the question about which academic works and profiles are acknowledged to exist in the scholarly information space. Given that institutions and individuals are capable of creating and sharing their works and profiles at Webspaces under their control (circa 1989, World Wide Web), it follows that DOI and ORCID are only social mechanisms to distinguish between what counts and what does not.

There is the traditional approach where an article goes through an opaque peer-review process; complies with the technical requirements of what constitutes a paper; and has the individual or their organisation paying the fees (to publish as well as to attend potential meetings), in addition to agreeing to the terms of the publisher. In this case, works are entitled to exist by means of centralised identifiers. If however this process is not met, the community does not necessarily consider what lays outside to be scientific knowledge. It goes without saying that publishing scientific knowledge on the Web, i.e., accessibly, and with different modes of engagement, is orthogonal to having a free and open, attributable scientific publication process. The requirements of publishers need not necessarily prevent us as individuals from publishing our work on our own websites. However, researchers are not occupying a vacuum, and contend with influences from many directions. These intertwined concerns make up the scholarly communication social machine, one part of which is that universities and libraries channel public funds for subscription fees so that researchers have read access to the articles, reinforcing the system. DOI and ORCID serve as top-down authoritarian constructs. Consequently, the type of data that can be fetched through these routes is fundamentally dependent on whatever the third-party publisher deems suitable for their own business and technical process.

The particular areas of research focus in Web Science is generally guided by conference organisation and journal committees, with the influence of senior researchers and their team’s interests, as well as the funding that is tied to that research. While this approach is practical and accommodates the community’s short-term needs and activities, it is evidently based on human-centred decision making as opposed to leveraging data-driven analysis of the field.

To further examine this, we observe that the topics of interest for calls for contributions is a reflection on what worked and did not from earlier experiences usually through town-hall discussions at conferences, as well as off-the-record discussions through different channels. This is valuable feedback from the community, however it could and should be accompanied by the possibility that new research challenges can be discovered by analysing directly data about what has already been studied. This would be a measurable observation which can feed back into the community, as well as providing actual data when it comes to requesting funding more legitimately.

Web Science can serve itself better if the freedom to innovate around how to best present and distribute research results is continuously encouraged. In order for the community to evolve using its own framework, the attempts at reformulating its conventions should be carefully considered and all centrally or top-down imposed restrictions should be thoroughly critiqued.

If articles and reviews accommodate machine-readable data (like Linked Data), the potential for reuse is increased. For example, if research articles capture their problem statements, motivation hypothesis, arguments, workflow steps, methodology, design, results, evaluation, conclusions, future challenges, as well as all inline semantic citations (to name a few) where they are uniquely identified, related to other data, and discoverable, then specialised software can be used to verify the article’s well-formedness with respect to the domain. In essence, this materialises the possibility of articles being executable towards reproduction of computational results. Similarly, user interfaces can manifest environments where readers can rerun experiments or observe the effects of changing the input parameters of an algorithm. This provides an important affordance for a more involving environment for the user, improves learnability of material, and supersedes the passive mode of simply reading about a proposed approach.

Innovation can also be encouraged by shortening the length of the research cycle. There are benefits to releasing early-work to the scientific community. For instance, having an early timestamp on some research can provide a strong signal on the origins of ideas and solutions, as well as having the value of practicality of receiving initial feedback from the community and public. Today, even some of the Open Access journal models slow down the scientific process (even if released to the public within first 2 years) when research is not immediately accessible.

We can examine social behaviours around writing articles for particular conferences and journals. For the sake of this discussion, we can refrain from judging the quality of the peer-reviews themselves since rich and poor reviews appear everywhere. This is especially true since detecting bias even in accurate research findings can be challenging.

One groupthink is a fixation on equations like: the number of accepted peer-reviewed articles over total contributions to a conference as a signifier for the venue’s importance in terms of academic credit. Lower the percentile tends to socially imply that a given venue is challenging for scholars to leave their scholarly mark. Unfortunately, such social perceptions create artificial bubbles.

To better understand these acceptance rates, we must think again in terms of the holistic social machine of scholarly communication. First, the number of acceptances are not solely based on the value of the contents of contributions as useful research output, but rather dependent on a number of other factors. The contributions that are accepted are ultimately subject to 1) the availability and suitability of the research tracks in a given year (decided, as we already mentioned, by the instinct of senior academics or whims of funding bodies), and 2) the number of articles allotted per volume or proceeding that the chairs are able to take on. Conferences that are collaborating with third-party publishers are instructed to commit to an optimal or preferred number of pages (or other criteria like number of words) for the conference series. These requirements are driven, again as we have already mentioned, by the business needs of for-profit organisations.

If venues were not subject to the constraints mentioned above, what would the information space be for the scientific output? Such endeavour would bring researchers to aim for venues or tracks that are most appropriate for their research area, and keep their focus solely on sound and reproducible research, as opposed to making it in to a journal that is artificially prestigious and imposed on researchers by third-party services. Perhaps we should find other standards by which to judge the quality of our research output aggregators instead. Criteria could include: are we permitted to present our work in the way that best conveys its message? Is my contribution deftly and conveniently situated alongside related work for passers-by? Do I have enough control to issue updates, or to engage with reviewers in context?

The Web Science community, like in other fields of research, goes above and beyond investing time, energy, and expertise into peer-reviews. With a few exceptions, reviewers are usually not publicly attributed and credited for their contributions. While the quality of peer-reviews may always be subject to question, they are nevertheless critical to providing some form of verification of scientific output. Hence, they should be captured, preserved, and environments should be created for further social engagement. Peer-review should not be a one-off event, but an ongoing conversation or collaboration. This desire amongst researchers is demonstrated by emerging services which allow academics to engage in pre and post publication peer-reviews and comments publicly to foster verifiable reviews and to keep track of retractions.

While most research publications in Web Science have their focus on positive results, as well as potentially reusable output (applications), the volume tends to pale in comparison to the negative or inconclusive results obtained during the research process. It is hypothesised that to get more out of science, negative results should as well be made visible and credited as they also serve useful information for other researchers to rapidly advance the field. Hence, workshops like Negative or Inconclusive Results in Semantic Web should be integral to the main research tracks or other workshops.

If we acknowledge an imminent new paradigm shift on the horizon, it is necessary to commit and exercise a new vocabulary in our academic endeavours. We should give importance to these new terms because they set our frame of mind, which sets the stage for our daily work. I would like to encourage new calls for contributions to welcome researchers to share their articles and supporting material on the Web:

s/submit/share

The archaic mental frame hinged on the idea that individuals submit their work, as in handing it over, dislocating themselves from what they have produced by giving up their future rights. There is no need to give up anything. It belongs to the creator, and they simply share it under their terms.

s/submission/contribution

A call by a venue does not merely get submissions of works. That is a misnomer. The process is much richer than that. Individuals make contributions to the field, pushing the boundaries of the collective understanding.

s/paper/article

The term paper is an obvious and dreadful artefact imposed by the pre-digital generation and archaic business models. Academics in Web Science tend to create articles (electronic documents if you will) that gets to reside somewhere on the Web, that can be discoverable and accessible.

Sooner or later we need a strategy to address questions like: What do we aim to accomplish? Who is trying to do it and for the benefit of whom? How do we know whether we are making progress and when we have succeeded?

Potential solutions to these questions and new challenges need to address the unspoken gap between researchers, academic and public institutions, as well as what ends up in the commons. It is not just about researchers publishing their work online, nor about conferences requesting semantically annotated contributions or operating with transparent peer-review. All of these things must move in combination. Here are some ideas that can be executed today:

Researchers

Researchers take full responsibility for the creation and publication of their work. This can be at any Webspace they put trust on or have sufficient authority over, e.g., personal site, institutions’ student and faculty pages. Making their work as Webby as possible e.g., linking their arguments and citations to others', interactive story-telling or experimentation possibilities for their readers. Researchers should take special care to ensure URI persistence and archiving of their work. See what your library or online archives (such as archive.org and archive.is) can do for you.

Conferences and journals

In person conferences and online journals should encourage and accept research publications which use native Web technologies. Support transparent and attributed peer-review process. Aggregation or proceedings of the events should be machine-friendly and archival measures should be taken in addition to that of researchers.

Public institutions

Given sufficient evidence of the Web Science community’s acknowledgement of research as well as its contributions (eg peer-reviews), academic institutions should officially acknowledge such work towards academic credit. Libraries should make the community’s output discoverable, catalogued, enriched, among many other things.