Skip to content

Make research_VENUS.load_VENUS() fault tolerant #25

@douglatornell

Description

@douglatornell

Original report by Doug Latornell (Bitbucket: douglatornell, GitHub: douglatornell).


load_VENUS() uses request.get() without any re-try log. It almost always works (and is fine for notebook use), but when a ConnectionError occurs in the nowcast system's use case it stops the automation path to pushing the results to the web (happene on 11dec15).

Idea:

  • Split load_VENUS() to separate getting the data from the ONC web service and parsing it into a pandas dataframe.
  • Retain a load_VENUS() function that gets data from the web for use in notebooks.
  • Change compare_VENUS() to accept either a buffer containing the web data, or a pandas dataframe as an arg.
  • Do the data download using nowcast.lib.get_web_data() in make_plots._make_research_plots() and pass the data into compare_VENUS().

Need to think about/explore binary vs. text data handling as nowcast.lib.get_web_data() presently only handles binary (response.content).

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions