Enhancing crop yield prediction with hybrid intelligence and advanced feature selection techniques

Abstract

Agriculture is vital to India s economy and sustains the livelihoods of over 60% of its population. As a major global producer of crops like rice, wheat, pulses, and cot- ton, agriculture plays a pivotal role in food security, rural development, and economic stability. Accurate crop yield prediction (CYP) is therefore essential not only to opti- mize agricultural productivity but also to guide policy decisions, ensure efficient supply chain management, and mitigate risks linked to environmental variability and climate change. This research introduces three distinct AI-driven models to enhance CYP per- formance. The first model, detailed in Chapter 3, presents a hybrid architecture that integrates Long Short-Term Memory (LSTM) and Deep Belief Networks (DBN), en- hanced by an Improved SMOTE oversampling technique and a fusion of feature rank- ing methods (Relief, Chi-square, and RFE). This model emphasizes temporal depen- dencies and feature importance for more reliable yield estimation. The second contri- bution, presented in Chapter 4, develops a deep ensemble approach combining CNN, Bidirectional GRU, and Deep Model Optimization (DMO) classifiers. It incorporates entropy-correlation-based feature engineering and leverages an Integrated Bird Swarm and Butterfly Optimization Algorithm (IBS-BOA) for optimal feature selection. This architecture is validated through convergence analysis, ablation studies, and statistical evaluation, demonstrating high prediction accuracy and robustness. The third model, described in Chapter 5, focuses on scalable computation. It employs Tanh normaliza- tion for preprocessing and uses the Apache Spark framework to handle large-scale data through master-slave architecture. An Enhanced Deep Fuzzy Clustering (DFC) algo- rithm partitions the dataset, while technical indicators (CCI, RSI, EMA, DEMA) are extracted as features. These are refined using a Modified SVM-RFE technique, and predictions are made using an Improved LinkNet classifier. This model stands out for its real-time readiness