---

title: Base dataset


keywords: fastai
sidebar: home_sidebar

summary: "Base class for dataset module."
description: "Base class for dataset module."
nb_path: "nbs/datasets/base.ipynb"
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/datasets/base.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
        
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="AbstractDataset">AbstractDataset<a class="anchor-link" href="#AbstractDataset"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="v1">v1<a class="anchor-link" href="#v1"> </a></h3>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="AbstractDataset" class="doc_header"><code>class</code> <code>AbstractDataset</code><a href="https://github.com/RecoHut-Projects/recohut/tree/master/recohut/datasets/base.py#L46" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>AbstractDataset</code>(<strong><code>args</code></strong>)</p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="v2">v2<a class="anchor-link" href="#v2"> </a></h3>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="AbstractDatasetv2" class="doc_header"><code>class</code> <code>AbstractDatasetv2</code><a href="https://github.com/RecoHut-Projects/recohut/tree/master/recohut/datasets/base.py#L139" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>AbstractDatasetv2</code>(<strong><code>args</code></strong>)</p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Dataset">Dataset<a class="anchor-link" href="#Dataset"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="SessionDataset">SessionDataset<a class="anchor-link" href="#SessionDataset"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="v1">v1<a class="anchor-link" href="#v1"> </a></h3>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="SessionDataset" class="doc_header"><code>class</code> <code>SessionDataset</code><a href="https://github.com/RecoHut-Projects/recohut/tree/master/recohut/datasets/base.py#L406" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>SessionDataset</code>(<strong><code>root</code></strong>, <strong><code>process_method</code></strong>, <strong><code>min_date</code></strong>=<em><code>None</code></em>, <strong><code>session_length</code></strong>=<em><code>None</code></em>, <strong><code>min_session_length</code></strong>=<em><code>None</code></em>, <strong><code>min_item_support</code></strong>=<em><code>None</code></em>, <strong><code>num_slices</code></strong>=<em><code>None</code></em>, <strong><code>days_offset</code></strong>=<em><code>None</code></em>, <strong><code>days_shift</code></strong>=<em><code>None</code></em>, <strong><code>days_train</code></strong>=<em><code>None</code></em>, <strong><code>days_test</code></strong>=<em><code>None</code></em>, <strong><code>data</code></strong>=<em><code>None</code></em>) :: <a href="/recohut/retailrocket.html#Dataset"><code>Dataset</code></a></p>
</blockquote>
<p>Session dataset base class.</p>
<p>Args:
    root (string): Root directory where the dataset should be saved.
    process_method (string):
        last: last day =&gt; test set
        last_min_date: last day =&gt; test set, but from a minimal date onwards
        days_test: last N days =&gt; test set
        slice: create multiple train-test-combinations with a sliding window approach
    min_date (string): Minimum date
    session_length (int): Session time length :default = 30 * 60 #30 minutes
    min_session_length (int): Minimum number of items for a session to be valid
    min_item_support (int): Minimum number of interactions for an item to be valid
    num_slices (int): Offset in days from the first date in the data set
    days_offset (int): Number of days the training start date is shifted after creating one slice
    days_shift (int): Days shift
    days_train (int): Days in train set in each slice
    days_test (int): Days in test set in each slice</p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="v2">v2<a class="anchor-link" href="#v2"> </a></h3>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="SessionDatasetv2" class="doc_header"><code>class</code> <code>SessionDatasetv2</code><a href="https://github.com/RecoHut-Projects/recohut/tree/master/recohut/datasets/base.py#L614" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>SessionDatasetv2</code>(<strong><code>root</code></strong>, <strong><code>column_names</code></strong>) :: <a href="/recohut/retailrocket.html#Dataset"><code>Dataset</code></a></p>
</blockquote>
<p>Dataset base class</p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="v3">v3<a class="anchor-link" href="#v3"> </a></h3>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="SessionDatasetv3" class="doc_header"><code>class</code> <code>SessionDatasetv3</code><a href="https://github.com/RecoHut-Projects/recohut/tree/master/recohut/datasets/base.py#L806" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>SessionDatasetv3</code>(<strong><code>root</code></strong>) :: <a href="/recohut/retailrocket.html#Dataset"><code>Dataset</code></a></p>
</blockquote>
<p>Dataset base class</p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="GraphData">GraphData<a class="anchor-link" href="#GraphData"> </a></h2>
</div>
</div>
</div>
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="GraphData" class="doc_header"><code>class</code> <code>GraphData</code><a href="https://github.com/RecoHut-Projects/recohut/tree/master/recohut/datasets/base.py#L970" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>GraphData</code>(<strong><code>data</code></strong>, <strong><code>shuffle</code></strong>=<em><code>False</code></em>, <strong><code>graph</code></strong>=<em><code>None</code></em>)</p>
</blockquote>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">train_data</span> <span class="o">=</span> <span class="p">([[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">]],</span> 
              <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">2</span><span class="p">])</span>

<span class="n">tds</span> <span class="o">=</span> <span class="n">GraphData</span><span class="p">(</span><span class="n">train_data</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">tds</span><span class="o">.</span><span class="n">generate_batch</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">tds</span><span class="o">.</span><span class="n">generate_batch</span><span class="p">(</span><span class="mi">2</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">tds</span><span class="o">.</span><span class="n">inputs</span><span class="p">)</span>

<span class="n">tds</span> <span class="o">=</span> <span class="n">GraphData</span><span class="p">(</span><span class="n">train_data</span><span class="p">,</span> <span class="n">shuffle</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">tds</span><span class="o">.</span><span class="n">generate_batch</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">tds</span><span class="o">.</span><span class="n">generate_batch</span><span class="p">(</span><span class="mi">2</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">tds</span><span class="o">.</span><span class="n">inputs</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">

<div class="output_subarea output_stream output_stdout output_text">
<pre>[array([0]), array([1]), array([2]), array([3]), array([4])]
[array([0, 1]), array([2, 3]), array([4])]
[[1 2 3]
 [2 3 4]
 [1 2 4]
 [2 3 0]
 [1 0 0]]
[array([0]), array([1]), array([2]), array([3]), array([4])]
[array([0, 1]), array([2, 3]), array([4])]
[[1 2 4]
 [2 3 4]
 [1 0 0]
 [2 3 0]
 [1 2 3]]
</pre>
</div>
</div>

</div>
</div>

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<hr>

</div>
</div>
</div>
</div>
 

