Tutorials

The Australian Digital Observatory Research Dashboard (ADOReD) is an interactive web based application designed to provide user friendly access to the ADO-API, to perform social media analysis. The dashboard allows users to interact with data present in the ADO databases, along with some of the topic modelling results and word embeddings that are generated by the ‘live’ machine learning pipelines. The dashboard works best a stable internet connection, and on a laptop or desktop computer, as it can be network...

The aggregation tab in ADOReD provides ability to aggregate over the social media collections in four different ways, which called different endpoints at the ADO-API. Location Time (by day) Seasonality Language Each aggregation is controlled by a start and an end date, to specify the period over which the aggregation is desired. Upon loading the aggregation tab, one can select the start and end date by using the widget in the top left corner. Once these value are selected, all...

The ADOReD topic graph tab provides users with an interactive way to examine the outputs of the ADO Topic Modelling pipeline. The pipeline is run every day, on a days worth of social media data, using BERTopic, to generate a list of topic clusters. These clusters simulate the trending topics of the given day, and each cluster is associated with 30 representative terms – from which the topic of the cluster can be extrapolated. Topic graph construction In order to track...

Two outputs of the ADO Topic Modelling pipeline (which is run daily, on a days worth of social media data) are the frequency of terms, as well as the development of word embeddings for the day. The ADO-API grants the ability to analyze the term frequency data, as well as interact with the word embedding models. The ADOReD Term analysis tab interacts with the API to visualize the outputs, and examine the semantic relationships between words in a certain time...

Some steps in this tutorial require Python, versions 3.8 and greater, to be installed along with the python packages requests, matplotlib and wordcloud. The Twitter API contains an endpoint which allows authorized users (details on how to attain access is listed here) to attain variety of information about the Tweet specified by the requested tweet ID or list of tweet IDs. There are a number of ways in which an interesting list of tweet ids may be attained, one of...

The full-text search functionality allows selecting social media posts (for the time being only for the Twitter collection) based on some post attributes: The attributes can be either equal or not equal to a value entered by the user, forming a `’rule’. Rules can be composed in groups joined by logical operators (‘and’, ‘or’, ‘not’). The expression can be changed manually by editing it before pressing the ‘Execute query’ button. A ‘Reset’ button allows for clearing the expression. The results...

ADOReD is harvesting Mastodon social media posts, and the social media post IDs can be downloaded from either the API or the Dashboard (for instance by executing text search queries). Once a text search is executed, some Mastodon post IDs can be downloaded in CSV format, for instance: The next logical step is to get the contents of said social media posts, which requires a bit of coding. To get the original social media posts, you first need to get...