This project is regarding generating the title and summary from the input paragraph. In this we have used T-5 Model which is text to text generation model.

Overview

Title and Summary Generation

This project is regarding generating the title and summary from the input paragraph. In this we have used T-5 Model which is text to text generation model.

The title and summary are being generated from pre-trained models.

For Title Generation we have used hugging face model: deep-learning-analytics/automatic-title-generation

For Summary Generation we have used hugging face model: sshleifer/distilbart-cnn-12-6

T-5 Model

T5 is an encoder-decoder model pre-trained on a multi-task mixture of unsupervised and supervised tasks and for which each task is converted into a text-to-text format. T5 works well on a variety of tasks out-of-the-box by prepending a different prefix to the input corresponding to each task, e.g., for translation: translate English to German: …, for summarization: summarize: ….

Architecture

alt text

Training Model

Training T5 is an encoder-decoder model and converts all NLP problems into a text-to-text format. It is trained using teacher forcing. This means that for training, we always need an input sequence and a corresponding target sequence. The input sequence is fed to the model using input_ids. The target sequence is shifted to the right, i.e., prepended by a start-sequence token and fed to the decoder using the decoder_input_ids. In teacher-forcing style, the target sequence is then appended by the EOS token and corresponds to the labels. The PAD token is hereby used as the start-sequence token. T5 can be trained / fine-tuned both in a supervised and unsupervised fashion.

One can use T5ForConditionalGeneration (or the Tensorflow/Flax variant), which includes the language modeling head on top of the decoder.

Unsupervised de-noising training

In this setup, spans of the input sequence are masked by so-called sentinel tokens (a.k.a unique mask tokens) and the output sequence is formed as a concatenation of the same sentinel tokens and the real masked tokens. Each sentinel token represents a unique mask token for this sentence and should start with <extra_id_0>, <extra_id_1>, … up to <extra_id_99>. As a default, 100 sentinel tokens are available in T5Tokenizer.

Supervised training

In this setup, the input sequence and output sequence are a standard sequence-to-sequence input-output mapping. Suppose that we want to fine-tune the model for translation for example, and we have a training example: the input sequence “The house is wonderful.” and output sequence “Das Haus ist wunderbar.”,

How to Run this project using editor:

  1. Make a virtual environment.
  2. Install all the packages of requirements.txt
  3. Make src as your working directory
  4. Run the file app.py

How to Run this project using Terminal:

  1. Make virtual environment using python3 venv {virtial env name}
  2. Activate virtual environment using source {virtial env name}/bin/activate
  3. Run app.py in src folder python3 app.py

If you are running this project using docker make sure change the path of title and summary model as: resources/summary_token

Open this Url after following above steps:

http://127.0.0.1:5001

You might also like...

Screenshot long text to get a short summary.

Screenshot long text to get a short summary.

tl;d(oc)r Screenshot long text ---- Output a short summary Utilize OCR on a screenshot to feed the captured text into a summarization CNN model. Requ

Apr 8, 2022

Discord bot that shows your information GM20 Book If you need help regarding GM20 book .

Discord bot that shows your information GM20 Book If you need help regarding GM20 book .

GM20-tools About | Screenshots | Installation | Usage | Disclaimer | Special Thanks | Support About The Discord bot that shows your information The gi

Aug 14, 2022

This repository includes projects I am working on regarding computer vision.

This repository includes projects I am working on regarding computer vision.

Computer-Vision This repository includes projects I am working on regarding computer vision. SIFT Feature Detection Image of best 100 matches detected

Aug 19, 2022

This app lets you search and sort talent by job title or relevant job descriptions from ecommurz talent list in real-time.

@ecommurz Talent Search Engine 💡 How it Works Before we dive into details, make sure to understand the basic concepts of semantic search which we exp

Jun 21, 2022

Scrape Google News articles - retrieves the title, article link, and OG Image for each news article

Scrape the latest Google News propaganda! Be sure to run multiple times per day as propaganda changes quickly! Retrieves Google News links using Selen

Aug 1, 2022

Google Maps Crawler takes Google Maps List and it scrape elements from all items such as: title, rating, reviews, location url, website,etc.

Google Maps Crawler Google Maps Crawler takes Google Maps List and it scrape elements from all items such as: title, rating, reviews, location url, we

Sep 15, 2022

A set of simple tools for splitting, merging, OP deletion, size compression, rewriting attributes and constants, OP generation, change opset, change to the specified input order, and RGB to BGR conversion for ONNX models.

A set of simple tools for splitting, merging, OP deletion, size compression, rewriting attributes and constants, OP generation, change opset, change to the specified input order, and RGB to BGR conversion for ONNX models.

simple-onnx-processing-tools A set of simple tools for splitting, merging, OP deletion, size compression, rewriting attributes and constants, OP gener

Sep 16, 2022

A very simple email spam bot. Not much to it. Just make sure you have python installed when you run it and have fun!

Simple-Email-Spam-Bot A very simple email bot I made with python. Not much to it. Just make sure you have python installed when you run it and have fu

Jul 7, 2022

Based on unique title like Tom Scott & Agung Hapsah's videos.

update-title-views Based on unique title like Tom Scott & Agung Hapsah's videos. Cara Pertama Setup YouTube API, bisa dibuka link tersebut: https://de

Aug 5, 2022
Owner
Anish Vijan
Passionate engineer with the ability to apply ML techniques and algorithm development to solve real world industry problems.
Anish Vijan
Multi-label Classification Model for English Scientific Literature: develop a multi-label classification model for scientific research literature based on the given metadata (title and abstract) of scientific research literature and corresponding hierarchical labels from a domain-specific topic taxonomy.

Multi-label Classification Model for English Scientific Literature: develop a multi-label classification model for scientific research literature based on the given metadata (title and abstract) of scientific research literature and corresponding hierarchical labels from a domain-specific topic taxonomy.

Yan Xinyi 2 Jul 14, 2022
WhatHitMe is a Python3 tool which provides the possible Groups that may have attacked you in an incident, based on specific Techniques and Software you have identified (MITRE ATT&CK)

WhatHitMe is a Python3 tool which provides the possible Groups that may have attacked you in an incident, based on specific Techniques and Software you have identified (MITRE ATT&CK). It also provides searches to known sources for these Groups to get a better insight of what you might had been facing!

Stratos Lontzetidis 6 Aug 5, 2022
Explore different chinese nlp tasks by using t5/mt5/t5-pegasus like text-classification, text-summary and so on.

T5 for nlp task: Two nlp-task about Text Classification(artificial prompt) and Text Summary based on T5/mt5/t5-pegasus had been tested Here. Notes: 从T

null 4 Sep 22, 2022
This project is an unofficial summary of the resources related to VALSE and its annual seminar. Its main purpose is to more facilitate your communication and learning, and we also welcome your additions and suggestions.

简体中文 | English Abstract Official website | Official Accounts | Blog The main purpose of the VALSE annual seminar is to provide a stage for deep academ

Xu Lin 7 Sep 21, 2022
It is a small project which uses transfer learning to classify the breed of the dog. The model used is MobileNetV2 which provides us 84.91% accuracy.

Dog-Breed-Classification It is a small project which uses transfer learning to classify the breed of the dog. The model used is MobileNetV2 from Tenso

Shishir Dwivedi 1 Aug 31, 2022
Python script which prints out a summary of your free slots from your Google calendar(s) so you can paste into a scheduling email.

print_google_calendar_availability Python script which prints out a summary of your free slots from your Google calendar(s) so you can paste into a sc

null 32 Jul 31, 2022
Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP

Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel order of RGB and BGR. Simple Channel Converter for ONNX.

Katsuya Hyodo 13 Aug 31, 2022
A tool to pin text to a specific set of texts. Useful to build a tool that takes any text input then infer which one of the anchor texts is the best one.

TextPinner A tool to pin text to a specific set of texts. Useful to build a tool that takes any text input then infer which one of the anchor texts is

Saifeddine ALOUI 1 Jun 27, 2022
A simple application regarding Student and Course Database Management System.

✔ STUDENT & COURSE MANAGEMENT SYSTEM This is an application regarding Student and Course Database Management System. REQUIREMENTS : python 3 tkinter m

Akash Rajak 1 Jul 1, 2022
Code for decision-making in long distance routes regarding the duration of driving and rest periods.

Model-based RL for route timing decisions Description Source code of the paper "Online Model-Based Reinforcement Learning for Decision-Making in Long

null 1 Jun 10, 2022