For this update, I immediately went to tackle two nagging issues. For one, the streamlit interface was simply too slow. For two, the time from query to query loading to answer from the LLM was simply too long. These combined factors created a lagging, slow user experience.
The biggest lesson here was more information on how different parameters of chunking and summarization affect the complexity of model usage. By increasing the similarity_top_k, we increase the amount of sources the engine can pull from. By implementing MMR query mode, we can increase the diversity of those sources. By decreasing the chunking sizes, we speed up the summarization of the sources and get our answers faster. Each of these has a trade-off that has become more apparent to me as I’ve worked through these issues.
Next up is adding this functionality directly into the Chronicler app! I’m looking for permission from the dev to ensure I’m not violating any licenses, but getting this tuned and in there is an exciting prospect for me!
]]>The objective of this project was to create a chatbot based on my personality.
The bot was a chatterbox library model, first trained on the provided english language corpus, and then further trained on a set of phrases / facts about myself to personalize the bot. It was then wrapped in a Flask app and deployed using PythonAnywhere, where it was accessible online. The bot has since been taken down to avoid upkeep costs.
The biggest challenge I ran into during this project was the time-constraint of the semester I completed it during. The TensorFlow model worked, but not well, and with time getting tight I pivoted to a simpler model.
This project provided valuable experience on the interworkings of neural nets for language processing, and the usage of new libraries. I also learned about Flask, and deploying models / applications where they can be accessible from (literally) anywhere. Throughout this project I also gave several presentations on my progress, improving my public speaking and ML presentation skills.
Given more time, within the constraints of the chatterbox library, I would have obtained a corpus of my own language, through text logs or emails, and train the model on that rather than the default corpus. The final model was prone to nonsense.
]]>The objective of this project was to create a better prediciton system for the scores of fantasy football players. Named for my son and integrated into a discord bot with the same name, its meant to represent an average ‘John’ in a fantasy football league.
This project used the NFLfastR API system to obtain the latest player statistical data, which was then ran through a linear regression model to predict a players point output for an upcoming week. The goal was to find a combination of features that would provide more accurate prediction then what was available on ESPN.
The biggest challenge of this project was finding the most important features to use in our model. Ultimately, the matchup ended up mattering most, that being a players past performance, and how good their next opponent was at limiting points allowed to that position group.
Most importantly with this project, I got valuable experience in deploying a trained model, with that model being readily available for use with the JohnFantasy discord bot. Players in the discord server are able to query the bot to find better predictions dynamically, with the model being retrained weekly on up-to-date data.
In the future, the model may be moved from a linear regression model to a neural net.
]]>The objective for this project was to enable a user of the JohnFantasy discord bot to freely query for a player’s stats and status. This was accomplished through the Named Entity Recognition of players names.
To make this work, the user first queries the bot using a discord endpoint, like !predict. They then type a sentence, etc., where they ask about a player. In some cases, this looks like “Tell me about
The efficency of this model is the biggest challenge. We don’t want the user waiting for multiple minutes in order to receive the results of their query, so the model must be lightweight, and the calls to the ESPN API need to be infrequent to reduce load.
This was another big exercise in integrating a finished model into a production environment, where efficiency and uptime are key. We also had to be accurate, as poor recognition led to garbage output or missed tags.
We are aiming to take this further in two ways. For one, we plan on obtaining or creating a corpus of NFL specific names / phrases, to ensure that we capture correctly as many of the tags as possible. We also aim to augment the free-text querying system to include intent classification, so that we can predict and subsequently output any number of player functions. As of right now, only player stat reporting is supported.
]]>As a frequent Game Master for mine and my friends Dungeons and Dragons games, I have amassed a large collection of lore pages within the DND Wiki-creating app Chronicler. As such, I aimed to create an assistant that could use the context of my hundreds of Wiki pages to answer questions about my created world, both for me and my players.
This project imported my wiki pages, tokenized and generated embeddings which were then stored for further use, and used a llama3 model with those embeddings as context to take in a question and output an answer, with citations. This process was then wrapped in a streamlit GUI to give the app a professional appearance.
The main challenge we ran into during this project was speed. Booting up the model and tokenizing and embedding the pages was slow, and several optimization techniques were needed, like storing the embeddings and changing the API-based model used. We also simplified the streamlit interface to speed up the startup process.
This project was a good experience in using API-based models for LLM tasks. I learned optimization techniques to get the most out of my hardware, and I learned valuable lessons in presenting models through the interface.
Going further, I believe there is still work to be done in speeding up the process. I am limited by hardware, but getting this hosted in the cloud would likely trivialize many of the issues I’ve had. I would also like to get this integrated into the Chronicler app, and discussions have been ongoing with the creator.
]]>The objective of Neurosynth was to create a real-time emotion to synth sound classifier.
Neurosynth was at its heart a convolutional neural net in Tensorflow, that used a vast input of labelled emotion pictures to classify a frame captured using the computer vision library OpenCV. This classification was then fed into pyaudio, where based on the emotion a synth sound played. The mapping of emotions followed typical convention, i.e. minor chords for sad or angry faces, and major chords for happy faces.
With this being my first CNN project, learning how to take in the facial input was difficult. Training required several forms of augmentation to reduce overfitting in the model. Procuring quality emotion images was also difficult, with several datasets being used.
This project provided valuable lessons in ensuring data quality, and tuning neural networks. Its very important with especially a CNN to strike the bias-variance balance. A deep model likely over-trains on the patterns in the training data, and a shallow model likely doesn’t learn deep enough patterns to make accurate predictions.
Going further in the project, we’d like to improve the real-time aspect of the predictions. As of right now, the image is only captured once every 1-2 seconds, which limits how fluid the synth outputs are. We’d also like to improve the variability of these outputs, making the chords flow into each other so it sounds musical.
]]>