https://davidan.dev/tags/nlp/ Recent content in nlp on David An Hugo -- gohugo.io en-us Wed, 01 Oct 2025 00:00:00 +0000 https://davidan.dev/posts/tokenization/ Wed, 01 Oct 2025 00:00:00 +0000 https://davidan.dev/posts/tokenization/ Lately, all we have heard about is tokenization and embeddings and the role they play in the greater LLM and AI ecosystem. These two concepts are one of the most fundamental concepts in language modeling and remain the foundation of the technology we interact with on a daily basis. In this article, we will cover some of the basics around tokenizing and embedding sequences of texts and the nuances of them. https://davidan.dev/research/nlpsearch/ Fri, 13 May 2022 00:00:00 +0000 https://davidan.dev/research/nlpsearch/ Building an NLP-Powered Repository for Cyber Risk Literature [Poster] David An, Linfeng Zheng, Zhiyu (Frank) Quan Abstract With the large and growing body of cyber risk literature, we see three major challenges faced by the actuarial research community: there is no context aware tool for finding cyber literature, no central repository of cyber risk resources, and a lack of accounting of literature trends. To address the abovementioned challenges, we propose to build a repository of cyber-risk articles with an NLP powered search tool that can easily be used by researchers to find relevant materials. https://davidan.dev/research/fadenet/ Wed, 12 May 2021 00:00:00 +0000 https://davidan.dev/research/fadenet/ FaDe-Net [Writeup] David An - AP Research Project Abstract The rapid development of social media and online news outlets has accelerated the spread of fake news across the internet. The accessibility and convenience of social media has further driven the drastic change of information consumption. As a consequence, fake news has become a significant concern because of 1) its inevitable exposure to large populations and 2) the potential to cause significant damage in modern society.