0 Introduction

In recent years, the rapid proliferation of social media platforms has generated an unprecedented volume of user-generated content ^[1]. This enormous data source offers an excellent chance to get insights into individual preferences and habits, as well as their geographic regions^[2-3]. Location prediction is very important in many different fields. A few applications that might greatly benefit from precise location forecasts are personalized suggestions, targeted advertising, urban planning, and public safety measures. Location prediction methods in the past depended on explicit check-ins or GPS data, necessitating active user involvement. Location prediction has become more accessible and unobtrusive because of social media network's ability to provide passive inference of location information^[4]. Social media platforms, which enable users to produce, share, and interact with material internationally, have emerged as the hub of interpersonal communication and information exchange. A complex tapestry of data describing user behaviors, interests, and interactions with locales is produced as a consequence of this pervasive social media use. Linking location prediction with social media data enables machine learning models to make use of the enormous volume of user-generated material, producing predictions that are more precise and contextually relevant^[5]. Through the use of machine learning (ML) based methodologies, location prediction via social media interaction has become an interesting study topic that tries to infer individuals' locations based on their activities and interactions on social media platforms^[6]. The availability of data and user participation were often issues with traditional techniques for location prediction^[7-8]. With social media's rapid expansion and extensive use, these platforms provide a wealth of up-to-date, varied data that surpasses the constraints of conventional approaches. The benefits of applying machine learning for location prediction are illustrated in Fig. 1. Social media involvement offers more detailed insights into users' preferences and behaviors, allowing ML models to perform better in capturing the complex correlations between activities and places. Machine learning algorithms provide a strong foundation for analyzing enormous amounts of social media data to find patterns and connections between places and activities^[9-12]. ML models may learn from labeled data to generate precise predictions and uncover hidden patterns in unlabeled data by using supervised and unsupervised learning approaches^[13-15]. While model modification increases the precision and effectiveness of location predictions, feature engineering enables the extraction of pertinent information from social media material.

Users often provide geographical details on social media, either manually or through GPS. However, manual data can be inaccurate, vague, general, or ambiguous. For instance, users might list fictional places or use names that refer to multiple locations, like "Washington". To better pinpoint a user's city-level location, it is suggested to analyze the content of their posts, using a probabilistic method that examines the terminology they use^[11-12]. The project aims to evaluate and compare different hybrid classification algorithms for predicting geolocation based on Twitter tweets^[13]. Previous studies have not compared these algorithms for this purpose^[14-15]. Identifying the best classification algorithm will enhance geolocation prediction accuracy, benefitting areas like surveillance, targeted advertising, crisis management, and demographic analysis.

Motivated by this, the key contributions of the paper are:

The paper introduces the location-aware attention LSTM (LAA-LSTM), a novel hybrid machine learning model designed specifically to predict the geographical location of tweets using their textual content. By utilizing an attention network, this model uniquely emphasizes the latitude and longitude features, which enhances its accuracy in identifying the user's tweet location. The performance of LAA-LSTM surpasses other existing machine learning approaches in the domain of tweet location prediction.

Therefore, main aspects of the proposed model includes:

·Integration of geographical information along with tweets differentiate it with other existing models.

·Integration of an attention network for hybridization of LSTM model.

·The model's application to social media content, particularly tweets, demonstrates its utility in analyzing and extracting meaningful information from short, often informal and varied text.

1 Literature Review

Using advanced AI and NLP techniques like sentiment analysis and Named Entity Recognition (NER), a system was developed by Sufi and Khalil^[16] to garner in-depth insights from social media feeds, specifically focusing on disaster-related content across 110 languages. This system was actively tested on live Twitter data from 28 September to 6 October 2021. Over the testing period, an impressive 67515 tweets from 39 unique languages were processed. Out of these, with a confidence level exceeding 70%, the system successfully extracted 9727 geographical indicators from the live Twitter data. This information was then integrated with disaster intelligence to highlight areas at potential risk of calamities. In terms of performance metrics, the system boasted an accuracy rate of 0.93, a recall rate of 0.88, and an F1-score of 0.90. The overall effectiveness of this fully autonomous disaster-tracking system was pegged at an impressive 97% accuracy. Murshed et al.^[17] developed a hybrid deep learning model known as DEA-RNN, which blends Elman RNNs and the Dolphin Echolocation Algorithm (DEA) to identify Content Behavior (CB) on Twitter. This model was shown to outperform many advanced algorithms in accuracy, precision, recall, F1-score, and specificity. Kumar and Singh^[18] presented a method using deep neural networks, including CNNs and LSTMs, for extracting location references from multilingual tweets. Their proposed CNN model, integrated with a conditional random field, demonstrated significant effectiveness in location-based applications. A location inference approach by Serere et al.^[19] enabled the prediction of tweet origin locations with high precision. They explored two datasets, demonstrating that tweet source distribution influences inference model success and showed promising results compared to state-of-the-art algorithms. Simanjuntak et al.^[20] focused on estimating the geographic location of Indonesian tweets using LSTM and BERT. By leveraging user profiles and tweets, they achieved remarkable accuracy, outperforming benchmark models. Mostafa et al.^[21] introduced a novel sentiment analysis-based model (Pre-HLSA) to predict Twitter users' residence locations using tweet attributes. Experimental results showed promising performance, reaching up to 85% accuracy. Mahajan et al.^[22] combined a CNN and a bidirectional LSTM to predict the geolocation of real-time tweets at the city level. Their model showcased significant improvements over baseline methods, achieving 92.6% accuracy. Azhar et al.^[23] proposed a deep learning model that integrates various data, including tweets and weather, to predict accidents. Their approach outperformed existing methods, improving detection accuracy up to 97.5%. Prasad et al.^[24] utilized the Twitter API to identify and categorize transportation disasters in Nigeria. They employed BERT along with AdamW optimizers, achieving an accuracy of 82%—an enhancement over the previous BERT-based method. Khan et al.^[25] evaluated machine learning techniques on a Weibo dataset. They suggested models like deep learning and gradient-boosted trees, with deep learning showing the highest accuracy at 99%. Inam et al.^[26] compared various machine learning and deep learning models to predict on-street parking availability. Their results indicated that simpler algorithms like Random Forest, Decision Tree, and K-Nearest Neighbor outperformed more complex ones like Multi-Layer Perceptron in accuracy. Their method considered multiple data sources, including weather, traffic flow, and pedestrian volume, to effectively predict parking slots. The study's solutions are scalable for bigger datasets and are apt for IoT-driven parking environments. Vohra et al.^[27] and Kateryna et al.^[28] have enriched our understanding of predicting geolocations from tweets. In 2023, Vohra et al.^[27] implemented a method that combined both geographical and textual features, enhancing the prediction accuracy. They discovered that this combined approach yielded better results than relying solely on textual data. On the other hand, Kateryna et al.^[28] harnessed deep learning techniques for their predictions. Their methodology surpassed many current strategies, even those based on conventional machine learning. The findings suggest the promising potential of deep learning in refining the accuracy of tweet-based geolocation predictions.

Therefore, the above review shows the application of AI and NLP, primarily in social media analysis, disaster response, and geolocation prediction. This makes the existing model more complex for handling linguistic information. Additionally, the quality of social media data, which can be inconsistent or incomplete, affects the accuracy of exisiting models. Those may result in bias result with existing models. Therefore, to handle these challenges, the proposed model presents an attention-based model with the hybrid approach of geographical information and textual (twitter data) information is used for predicting location which is quite appropriate and efficient.

2 Overview of Attention Network

An attention network is a type of deep-learning model with an attention mechanism^[29]. It is generally composed of an embedding layer, and an encoding layer, with an attention mechanism. The input embedding layer converts input such as words or text into fixed-size vectors where each embedding matrix is created corresponding to each input vector. Then, the encoding layer is added relevant to the position which adds information of each input position to the sequence. This is added with an embedding matrix to track the order of input tokens. The attention mechanism is the core concept of the network which is designed with three basic components such as query (Q), key (K), and value (V). The input vector is mapped in the form of Q, K, V. For this mapping, linear transformation is used. Applying dot product over these components' attention weights is evaluated^[30].

3 Methodology Used

In this paper, location prediction is proposed using Twitter geolocations. Prediction is based on determining user's location based on their tweets. In this paper, we have used the GeoText^[31] dataset and GeoCov19^[32] from Twitter with geographical hints either in text or distinct messages. These tweets, once processed and cleaned, can be fed into machine learning algorithms for analysis. After training and testing, these algorithms can predict user location based on textual clues such as city names or regional terms. Geolocation-labeled tweet datasets can be used to train classifiers to predict specific regions like North, South, East, West, North-East, North-West, South-East, and South-West. Once trained, these classifiers can effectively pinpoint a user's geographical region. The entire working model is presented below in Fig. 2.The working steps are presented in Fig. 3.

3.1 Data Collection

For the research conducted in this paper, data was sourced from two primary datasets: GeoText^[31] and GeoCov19^[32]. This dataset comprises tweets that carry geographical hints within their textual content. These hints could be explicit mentions of locations, landmarks, or other location-specific terminologies that give away the geographical context of the tweet. Both datasets are utilized to extract tweets that offer either explicit or implicit cues about their originating location, ensuring a robust and diverse set of data for training and evaluating the location-aware attention LSTM (LAA-LSTM) model. The dataset used in the research, comprising GeoText^[31] and GeoCov19^[32], is appropriate for training and evaluating a location-aware LSTM model as it includes tweets with both explicit and implicit geographical hints. The data is publicly available. However, potential biases due to Twitter's user demographics and the specific timeframe of data collection should be considered.

3.2 Data Preparation

The dataset preparation for this study involved several crucial steps to ensure that the data was ready for model training and evaluation.

1) Extraction of latitude and longitude: Relevant geographical coordinates (latitude and longitude) were extracted from the datasets. These coordinates give explicit location information about where the tweet originated or what location it referred to.

2) Data cleaning: This is a general process of identifying and correcting (or removing) errors and inconsistencies in data to improve its quality. It ensures that the data is accurate, valid, consistent, relevant, and complete.

3) Tweet data cleaning: Specific to the tweets, this process likely involved the removal of URLs, mentions, hashtags, emojis, and other non-essential elements. The aim was to retain only the meaningful text content of the tweets for analysis.

4) Tweet data normalization: Normalization typically refers to the process of bringing or transforming the data into a common format or standard. In the context of tweets, this could involve converting all text to lowercase, stemming or lemmatizing words (reducing them to their base or root form), and replacing slang or abbreviations with their full forms.

5) Removing redundant data: Any duplicate or repetitive tweets and data entries were identified and removed. This ensures that the dataset is streamlined and that the model is not overfitting or being biased due to repeated information.

These steps collectively ensured that the dataset was not only clean but also structured and optimized to predict tweet locations using the LAA-LSTM model.

3.3 Feature Extraction

First of all, the tweets are tokenized. Tokenization is the process of converting text into sequences of integers, where each integer corresponds to a specific word or token in a dictionary. The tweets are then converted into sequences of integers using the texts-to-sequences method. These sequences are padded to ensure they have a consistent length. This is essential for feeding them into neural networks, where input shape consistency is needed.

The "region" column of the dataset is converted as a label for the training of the learning model.

3.4 Learning and Pattern Evaluation

The hybrid location aware-attention LSTM (LAA-LSTM) model is based on an attention mechanism with an LSTM layer and merges its output with another input. The attention class implements the attention mechanism with longitude and latitude information. These are dense layers that are used to compute the attention weights and the context vector from a given hidden state and features. The main model starts with an embedding layer that transforms input text sequences into dense vectors. This is followed by an LSTM layer with LSTM units that output sequences. The attention layer then computes a context vector by focusing on relevant parts of the LSTM output. The context vector and the lat-long input are concatenated and passed through a dense layer. The architecture of the proposed LAA-LSTM model is presented below in Fig. 4.

Finally, the model predicts a region using a Random Forest(RF) classifier. The model takes in a sequence of text and a latitude-longitude pair and outputs a distribution over regions. The RF classifier is an ensemble learning method primarily used for classification (and regression). Let us delve into its mathematical description. The RF algorithm works by building multiple decision trees during training. Each of these trees is built using a subset of the data and a subset of the features. The mathematical description behind each decision tree involves recursively splitting the data to maximize the homogeneity of the target variable in each resulting subset. The most common split criterion for classification tasks is the entropy.

$ \text { Entropy }=-p \log (p)-(1-p) \log (1-p) $

(1)

For a new data point, the prediction in a RF classifier is the majority vote (mode) of the predictions of individual trees.

$ \hat{y}=\operatorname{mode}\left(\hat{y}_1, \hat{y}_2, \cdots, \hat{y}_n\right) $

(2)

Feature importance can be computed using the average Gini impurity decrease (for classification) caused by each feature across all trees.

$ \text { Feature Importance }=\frac{1}{N} \sum\limits_{t=1}^n \Delta \mathrm{emtropy}(f, t) $

(3)

where, the number of trees in the forest is represented as N, and Δemtropy(f, t) represents the decrease of entropy of feature f in tree t. In essence, a RF algorithm provides a powerful and flexible tool for classification by leveraging the strength of multiple decision trees, reducing overfitting, and providing feature importance metrics.

4 Results and Discussion

In this paper, the entire model is simulated on the Python platform over Google colab with the facility of Tesla P100-PCIE GPU. The performance was evaluated using the following parameters:

$ \text { Accuracy }=\frac{(\mathrm{TP}+\mathrm{TN})}{(\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN})} $

(4)

$ \text { Precision }=\frac{(\mathrm{TP})}{(\mathrm{TP}+\mathrm{FP})} $

(5)

$ \text { Recall }=\frac{(\mathrm{TP})}{(\mathrm{TP}+\mathrm{FN})} $

(6)

$ \text { F1-score }=\frac{(2 \times \text { Precision } \times \text { Recall })}{(\text { Precision }+ \text { Recall })} $

(7)

where TP is the true positive, TN is the true negative, FP is false positive and FN is false negative.

Table 1 presents the comparative result analysis of the presented Hybrid model and baseline model i.e., LSTM. The Hybrid (LAA-LSTM) model outperforms the LSTM model in terms of weighted averages across all metrics. The weighted average accuracy for hybrid is 75%, while the LSTM model has an accuracy of 54%. For the regions North-East, South-East, and South-West, the Hybrid model demonstrates superior accuracy over the LSTM model. The wighted average accuracy for Hybrid is 75%, while the LSTM model has an accuracy of 54%. For the North-East and South-West regions, the precision for both models is relatively close. Still, the LSTM slightly edges out in the North-East, whereas the Hybrid is better in the South-West. For the North-West region, the Hybrid model has a significantly higher precision (63.00%) compared to the LSTM model's precision (76.00%). In the South-East region, the Hybrid model's precision (46.00%) is vastly superior to the LSTM model (19.00%). On the weighted average, the Hybrid model again has a better precision (74.00%) than the LSTM model (57.00%). The Hybrid model outperforms the LSTM model in terms of recall in the regions North-East, South-East, and South-West. However, for the North-West region, the LSTM's recall is notably low at 7.00%, while the Hybrid model achieves a recall of 49.00%. On the weighted average, the Hybrid model's recall (75.00%) is substantially higher than the LSTM model's recall (54.00%). The Hybrid model exhibits a higher F1-score in all regions except North-West compared to the LSTM model. In the weighted average metric, the Hybrid model's F1-score (74.00%) is significantly superior to the LSTMCs (47.00%). Therefore, it is concluded that overall, the Hybrid (LAA-LSTM) model is generally superior to the LSTM model across most regions and metrics. Given this, for applications that prioritize consistent performance across different regions, the Hybrid (LAA-LSTM) model would be a more favorable choice based on the presented data.

Fig. 5 presents the confusion matrix of LSTM and the proposed hybrid LAA-LSTM model.

Fig. 6 presents the results for two models, LSTM and LAA-LSTM. The LAA-LSTM model has a 21% higher accuracy than the LSTM model. The LAA-LSTM model has a 17% higher precision, 21% higher recall, and 27% higher F1-score than the LSTM model. Therefore, the proposed hybrid (LAA-LSTM) model consistently outperforms the traditional LSTM model in all the metrics provided.

Fig. 7 presents comparative analysis of proposed model with state-of-art several methods. The figure states that LR^[34] achieved 31.00%, RF^[34] achieved 34.00%, NB^[34] achieved 44.00%, k-NN^[34] achieved 29.00%, RF^[33] achieved 68.32%, NB^[33] achieved 50.40%, SVM^[33] achieved 61.26%, LSTM achieved 54.00%, and LAA-LSTM achieved 75.00%. From the figure, it is inferred that the proposed hybrid LAA-LSTM method appears to be the best-performing model as compared to existing machine-learning approaches for location prediction using social media content.

5 Conclusions

Location prediction on social media, using Machine Learning (ML) to predict geographical coordinates or location labels from content, is a significant research area. The paper introduces a novel ML model, location-aware attention LSTM (LAA-LSTM). This LSTM model emphasizes the latitude and longitude features through an attention network to pinpoint the user's tweet location. The results indicate that this approach outperforms other existing machine-learning methods in tweet location prediction. The proposed model shows a 10% improvement in accuracy as compared to others. While location prediction has broad applications, including personalized recommendations and disaster response, future research must tackle challenges like data privacy and the dynamic nature of social media to fully harness this technology. In future, this work will be extended in multi-modular architecture for location prediction using image and textual data.