Krptkn.UrlQueue (Кропоткин Framework v0.1.0) View Source
This module defines a GenServer that acts as a global queue so that consumers can push new URLs that they find and producers can pop them to extract the HTML.
Link to this section Summary
Functions
Returns a specification to start this module under a supervisor.
Removes all elements from the url queue.
Removes all elements from the ETS table that holds the already visited URLs.
Pauses the URL queue so it will stop serving new URLs.
Pops a URL from the queue.
Pushes a URL to the queue and saves it on the ETS table (to avoid visiting twice a URL).
Resumes the URL queue if it was paused.
Checks the state of the framework.
Link to this section Functions
Returns a specification to start this module under a supervisor.
See Supervisor.
Removes all elements from the url queue.
Returns :ok.
Examples
iex> Krptkn.UrlQueue.clear_queue()
:okRemoves all elements from the ETS table that holds the already visited URLs.
Returns :ok.
Examples
iex> Krptkn.UrlQueue.clear_visited()
:okPauses the URL queue so it will stop serving new URLs.
Returns :ok.
Examples
iex> Krptkn.UrlQueue.pause()
:okPops a URL from the queue.
Returns a string with a URL.
Examples
iex> Krptkn.UrlQueue.pop()
"https://stallman.org/"Pushes a URL to the queue and saves it on the ETS table (to avoid visiting twice a URL).
Returns :ok.
Examples
iex> Krptkn.UrlQueue.push("https://stallman.org/")
:okResumes the URL queue if it was paused.
Returns :ok.
Examples
iex> Krptkn.UrlQueue.resume()
:okChecks the state of the framework.
Returns :running or :stopped.
Examples
iex> Krptkn.UrlQueue.state()
:running