seriously funny

github

Pratyush Shukla  • Peerlist

This project aims to check the robustness of the CLIPcap Image Captioning Model using an adversarial attack from an Encoder-Decoder architecture-based model on the MSCOCO 2014 dataset. The Encoder model is based on ResNet-101, and the Decoder model uses LSTM along with an attention mechanism and Beam Search.

Adversarial Attack on Image Captioning Model

Synthesized 160 similar data points on Zia’s dataset by applying Applied Synthetic Minority Oversampling Technique using NumPy and scikit-learn APIs. 
Analyzed results of 15+ popular Machine Learning models by using scikit-learn and XGBoost APIs. 
Developed 2 optimization algorithms in MATLAB: Energy-Efficient Bat and Cost-Efficient Bat for hyperparameter tuning of Neuro-Fuzzy systems, boosting prediction accuracy to 99.7%.

Agile Project Effort and Cost Estimation

Vision Transformers are increasingly embedded in industrial systems due to their superior performance, but their memory and power requirements make deploying them to edge devices a challenging task. Hence, model compression techniques are now widely used to deploy models on edge devices as they decrease the resource requirements and make model inference very fast and efficient. But their reliability and robustness from a security perspective is another major issue in safety-critical applications. Adversarial attacks are like optical illusions for ML algorithms and they can severely impact the accuracy and reliability of models. In this work we investigate the transferability of adversarial samples across the SOTA Vision Transformer models across 3 SOTA compressed versions and infer the effects different compression techniques have on adversarial attacks.

Attacking Compressed Vision Transfers

Using PyTorch, Detectron2, and OpenCV APIs, developed a Person Detector model using ResNet50-FPN base network tuned to 95% accuracy, Instance Segmentation model using Mask R-CNN model with ResNet50-FPN backbone tuned to 98% accuracy and using this pre-trained network to detect balloons tuned to 99% accuracy.

Balloon Detector

The project aims to test the performance of Julia’s Deep Learning ecosystem using Flux.jl package against Python’s PyTorch and assess whether Julia’s motto of “Looks like Python, feels like Lisp, runs like C/Fortran” is justified. The results are based on detailed benchmarking analysis of PyTorch and Flux.jl performances on Neural Network training and hyperparameter optimization over the High-Performance Computing system.

Benchmarking Comparison of Julia and Python's Deep Learning 

Utilized DeFiLlama-Curl API to extract protocol data. Performed EDA to observe market trends of top-performing protocols. 
Performed Feature Engineering and employed Prophet library to predict the market trend for the future 3 months tuned for 95% accuracy.

Blockchain Protocol Analysis

DeFiLlama-Curl

Developed a human face detector and a dog face detector using pretrained VGG-16. Further developed and trained a model from scratch and using transfer learning to compare the accuracy results for detecting dog breeds. Finally, created a pipeline for incorporating the aforementioned algorithms into a single application. There are two variants of the classifier, in TensorFlow using Keras and in PyTorch.

Dog Breed Classifier

Performed EDA by analyzing factors affecting insurance charges. Performed Feature Engineering by developing cost features. 
Compared Linear, Polynomial, Ridge, Gradient Boost, XGBoost, and CatBoost regression models to predict clients’ insurance charges. 
Tuned the hyperparameters of the Gradient Boost model to obtain 85.44% accuracy.

Insurance Modelling

Evaluate the impact of the quarterly earnings report on stock price movement. Developed in C++ using standard libraries and custom plotting library.