Tag Archives: iNaturalist

India has 100k records on iNaturalist

22 Apr

Biodiversity citizen scientists use iNaturalist to post their observations with photographs. The observations are then curated there by crowd-sourcing the identifications and other trait related aspects too. The data once converted to “research grade” is passed on to GBIF as occurrence records.

Exciting news from India in 3rd week of April 2019 is:

Being interested in biodiversity data visualizations and completeness, I was waiting for 100k records to explore the data. Here is what I did and found out.

Step 1: Download the data from iNaturalist website. Which can be done very easily by visiting the website and choosing the right options.

https://www.inaturalist.org/observations?place_id=6681

I downloaded the file as .zip and extracted the observations-xxxx.csv. [In my case it was observations-51008.csv].

Step 2: Read the data file in R

library(readr)
observations_51008 <- read_csv("input/observations-51008.csv")

Step 3: Clean up the data and set it up to be used in package bdvis.

library(bdvis)

inatc <- list(
  Latitude="latitude",
  Longitude="longitude",
  Date_collected="observed_on",
  Scientific_name="scientific_name"
)

inat <- format_bdvis(observations_51008,config = inatc)

Step 4: We still need to rename some more columns for ease in visualizations like rather than ‘taxon_family_name’ it will be easy to have field called ‘Family’

rename_column <- function(dat,old,new){
  if(old %in% colnames(dat)){
    colnames(dat)[which(names(dat) == old)] <- new
  } else {
    print(paste("Error: Fieldname not found...",old))
  }
  return(dat)
}

inat <- rename_column(inat,'taxon_kingdom_name','Kingdom')
inat <- rename_column(inat,'taxon_phylum_name','Phylum')
inat <- rename_column(inat,'taxon_class_name','Class')
inat <- rename_column(inat,'taxon_order_name','Order_')
inat <- rename_column(inat,'taxon_family_name','Family')
inat <- rename_column(inat,'taxon_genus_name','Genus')

# Remove records excess of 100k
inat <- inat[1:100000,]

Step 5: Make sure the data is loaded properly

bdsummary(inat)

will produce some like this:

Total no of records = 100000 

 Temporal coverage...
 Date range of the records from  1898-01-01  to  2019-04-19 

 Taxonomic coverage...
 No of Families :  1345
 No of Genus :  5638
 No of Species :  13377 

 Spatial coverage ...
 Bounding box of records  6.806092 , 68.532  -  35.0614769085 , 97.050133
 Degree celles covered :  336
 % degree cells covered :  39.9524375743163

The data looks good. But we have a small issue, we have some records from year 1898, which might cause trouble with some of our visualizations. So let us drop records before year 2000 for the time being.

inat = inat[which(inat$Date_collected > "2000-01-01"),]

Now we are ready to explore the data. First one I always like to see is geographical coverage of the data. First let us try it at 1 degree (~100km) grid cells. Note here I have Admin2.shp file with India states map.

mapgrid(inat,ptype="records",bbox=c(60,100,5,40),
        shp = "Admin2.shp")

mapgrid1_c

This shows a fairly good geographical coverage of the data at this scale. We have very few degree cells with no data. How about fines scale though? Say at 0.1 degree (~10km) grid. Let us generate that.

mapgrid(inat,ptype="records",bbox=c(60,100,5,40),
        shp = "Admin2.shp",
        gridscale=0.1) 

mapgrid2_c

Now the pattern is clear, where the data is coming from.

To be continued…

References

Mapping Biodiversity data on smaller than one degree scale

23 Feb

Guest Post by Enjie (Jane) LI

I have been using bdvis package (version 0.2.9) to visualize the iNaturalist records of RAScals project (http://www.inaturalist.org/projects/rascals).

Initially, the mapgrid function in the bdvis version 0.2.9 was written to map the number of records, number of species and completeness in a 1-degree cell grid (~111km x 111km resolution).

I applied this function to the RASCals dataset, see the following code. However, the mapping results are not satisfying. The 1 degree cells are too big to reveal the details in the study areas. Also, the raster grid was on top the basemap, which makes it really hard to associate the mapping results with physical locations.

library(rinat)
library(bdvis)

rascals=get_inat_obs_project("rascals")
conf <- list(Latitude="latitude",
             Longitude="longitude",
             Date_collected="Observed.on",
             Scientific_name="Scientific.name")
rascals <- format_bdvis(rascals, config=conf)
## Get rid of a record with weird location log
rascals <- rascals[!(rascals$Id== 4657868),]
rascals <- getcellid(rascals)
rascals <- gettaxo(rascals)
bdsummary(rascals)

a <- mapgrid(indf = rascals, ptype = "records",
             title = "distribution of RASCals records",
             bbox = NA, legscale = 0, collow = "blue",
             colhigh = "red", mapdatabase = "county",
             region = "CA", customize = NULL)

b <- mapgrid(indf = rascals, ptype = "species",
              title = "distribution of species richness of RASCals records",
              bbox = NA, legscale = 0, collow = "blue",
              colhigh = "red", mapdatabase = "county",
              region = "CA", customize = NULL)

rascals01

rascals02

I contacted developers of the package regarding these two issues. They have been very responsive to resolve them. They quickly added the gridscale argument in the mapgrid function. This new argument allows the users to choose scale (0.1 or 1). The default 1-degree cell grid for mapping.

Here are mapping results from using the gridscale argument. Make sure you have bdvis version 0.2.14 or later.

c <- mapgrid(indf = rascals, ptype = "records",
             title = "distribution of RASCals records",
             bbox = NA, legscale = 0, collow = "blue",
             colhigh = "red", mapdatabase = "county",
             region = "CA", customize = NULL,
             gridscale = 0.1)

d <- mapgrid(indf = rascals, ptype = "species",
             title = "distribution of species richness of RASCals records",
             bbox = NA, legscale = 0, collow = "blue",
             colhigh = "red", mapdatabase = "county",
             region = "CA", customize = NULL,
             gridscale = 0.1)

rascals03

rascals04

We can see that the new map with a finer resolution definitely revealed more information within the range of our study area. One more thing to note is that in this version developers have adjusted the basemap to be on top of the raster layer. This has definitely made the map easier to read and reference back to the physical space.

Good job team! Thanks for developing and perfecting the bdvis package.

References

package bdvis is on CRAN

8 May

We are happy to announce that package bdvis is on CRAN now. http://cran.r-project.org/web/packages/bdvis/index.html

bdvis: Biodiversity Data Visualizations

Biodiversity data visualizations using R would be helpful to understand completeness of biodiversity inventory, extent of geographical, taxonomic and temporal coverage, gaps and biases in data.

As part of Google Summer of Code 2014, we hope to make progress on the development of this package and the proposed additions are posted here.

If you have never used package bdvis the following code will give you a quick introduction of the capabilities of the package.

First to install the package

install.packages("bdvis")
library(bdvis)
# We use rinat package to get some data from
# iNaturalist project
# install.packages("rinat")
library(rinat)

Now let us get some data from iNaturlist project ReptileIndia

inat=get_inat_obs_project("reptileindia")
239  Records
0-100-200-300

We need to convert the data in bdvis format.

  • Use fixstr function to change names of two fields.
  • Use getcellid function to calculate grid numbers for each records with coordinates.
  • Use gettaxo function to fetch higher taxonomy of each record. This function will take some time to run and might need some human interaction to resolve names depending on the data we have.
# Function fixstr is now replaced with format_bdvis
# inat=fixstr(inat,DateCollected="Observed.on",SciName="Scientific.name")
inat=format_bdvis(inat,source='rinat')
inat=getcellid(inat)
inat=gettaxo(inat)

Our data is ready for trying out bdvis functions now. First a function to see what data we have.

bdsummary(inat)

The output should look something like this:

 Total no of records = 239 
 Date range of the records from  2004-07-31  to  2014-05-04 
 Bounding box of records  5.9241302618 , 72.933495  -  
30.475012 , 95.6058760174 
 Taxonomic summary... 
 No of Families :  16 
 No of Genus :  52 
 No of Species :  117 

Now let us generate a heat-map with geography superimposed. Since we know this project is for Indian subcontinent, we list the countries we need to show on the map.

mapgrid(inat,ptype="records",
        bbox=c(60,100,5,40),
        region=c("India","Nepal","Bhutan",
                  "Pakistan","Bangladesh",
                   "Sri lanka", "Myanmar"),
        title="ReptileIndia records")
ReptileIndia mapgrid

ReptileIndia mapgrid

For temporal visualization we can use tempolar function with plots number of records on a polar plot. The data can be aggregated by day, week or month.

tempolar(inat, color="green", title="iNaturalist daily",
         plottype="r", timescale="d")
tempolar(inat, color="blue", title="iNaturalist weekly",
         plottype="p", timescale="w")
tempolar(inat, color="red", title="iNaturalist monthly",
         plottype="r", timescale="m")
ReptileIndia tempolar daily

ReptileIndia tempolar daily

ReptileIndia tempolar weekly

ReptileIndia tempolar weekly

ReptileIndia tempolar monthly

ReptileIndia tempolar monthly

Another interesting temporal visualization is Chronohorogram. This plots number of records on each day with colors indicating the value and concentric circles for each year.

chronohorogram(inat)
ReptileIndia chronohorogram

ReptileIndia chronohorogram

And finally for taxonomic visualization we can generate a tree-map of the records. Here the color of each box indicates number of genus in the family and the size of the box indicates proportion of records in the data set of each family.

taxotree(inat)
ReptileIndia taxotree

ReptileIndia taxotree

The large empty box at bottom center indicates there are several records which are not identified at family level.

Check the post GSoC Proposal 2014: package bdvis: Biodiversity Data Visualizations for what to expect in near future and comments and suggestions are always welcome.

Package rinat use case: map of iNaturalist project

11 Mar

iNaturalist projects are collection of records posted on iNatualist. Now that we have a R package rinat from rOpenSci I thought of playing around with the data. Here is a function I wrote, to quickly map all the records of a project using ggmap package.

library(ggmap)
library(rinat)

inatmap <- function(grpid){
  data1=get_inat_obs_project(grpid, type = "observations")
  data1=data1[which(!is.na(data1$Latitude)),]
  map <-get_map(location =c(min(data1$Longitude),
                            min(data1$Latitude),
                            max(data1$Longitude),
                            max(data1$Latitude)),
                messaging = FALSE)
  p <-ggplot()
  p= ggmap(map)+geom_point(data=data1,
                           aes(x=as.numeric(Longitude),
                               y=as.numeric(Latitude)))
  p
}

We can used get_inat_obs_project function from rinat package to get all the observation from the specified project. get_map function form ggmap package to download google maps base layer and ggplot function form ggplot2 package to actually plot the map with points.

Now call to the function with a group name will produce a map with all the records in the project.

inatmap("birdindia")

inatmap_birdindia

We can use other ggplot options to add title, legend etc. to the map. This is just a simple example.