89 lines
2.2 KiB
Markdown
89 lines
2.2 KiB
Markdown
# Story Point Estimator
|
|
|
|
This repository contains code for training and evaluating and exporting models for story point estimation.
|
|
|
|
The dataset used for training is the [IEEE TSE2018 dataset](https://github.com/jai2shukla/JIRA-Estimation-Prediction/tree/master/storypoint/IEEE%20TSE2018/dataset) which contains user stories from 16 open-source projects from 9 different organizations.
|
|
|
|
|
|
Currently, the following models are supported:
|
|
- [x] [DistilBERT](https://arxiv.org/abs/1910.01108)
|
|
|
|
|
|
## Setup
|
|
|
|
#### Create and activate conda environment
|
|
|
|
```bash
|
|
conda create --name spestimator python=3.8
|
|
conda activate spestimator
|
|
```
|
|
|
|
#### Install dependencies
|
|
|
|
```bash
|
|
pip install torch==2.4.1 --index-url https://download.pytorch.org/whl/cu124
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
#### Setup dotenv
|
|
|
|
Create a `.env` file in the root directory and add the following environment variables:
|
|
|
|
```bash
|
|
DATA_PATH=<path to data directory>
|
|
```
|
|
|
|
|
|
## Usage
|
|
|
|
#### Train models for story point estimation
|
|
|
|
See training options
|
|
|
|
```bash
|
|
python train.py --help
|
|
```
|
|
|
|
Train a model with your desired options while also seing validation results
|
|
|
|
```bash
|
|
python train.py <options>
|
|
```
|
|
|
|
Users can see in the experiments folder the results inside a subfolder with their experiment's name. The folder contains: <br>
|
|
- a checkpoints folder where the results of each epoch and their according performance metrics are stored <br>
|
|
- a params.json file with the experiment's parameters <br>
|
|
- a log file with the experiment's logs regarding the training process <br>
|
|
|
|
|
|
#### Evaluate existing models
|
|
|
|
See evaluation options
|
|
|
|
```bash
|
|
python eval.py --help
|
|
```
|
|
|
|
Evaluate a model with your desired options
|
|
|
|
```bash
|
|
python eval.py <options>
|
|
```
|
|
|
|
|
|
#### Export models
|
|
|
|
See export options
|
|
|
|
```bash
|
|
python export.py --help
|
|
```
|
|
|
|
Export a model with your desired options
|
|
|
|
```bash
|
|
python export.py <options>
|
|
```
|
|
|
|
Exports the model and tokenizer to the specified directory. The <b>model</b> is stored in a <b>.safetensors</b> file (for python users) and an <b>.onnx</b> file for other languages, and the <b>tokenizer</b> is stored in a <b>vocab.txt</b> file and 4 json files: <b>config.json, special_tokens_map.json, tokenizer_config.json, and tokenizer.json</b>.
|