Visible light communication (VLC) has emerged as a compelling alternative to conventional radio frequency (RF) technologies, offering benefits such as high bandwidth availability, license-free spectrum, improved physical layer security, and immunity to electromagnetic interference [1]. These advantages make VLC particularly attractive for dense indoor environments [2]. However, VLC systems remain highly dependent on the availability of a clear line-of-sight (LoS) between the transmitter and the receiver, which can be compromised by user mobility, random device orientation, and dynamic obstacles [3]. These impairments result in severe degradation of link quality and overall system performance.

To address these limitations, intelligent reflecting surfaces (IRS) have recently been introduced in VLC systems as a promising solution to improve channel robustness [4]. An IRS consists of a large number of passive optical elements capable of dynamically reflecting incident light towards desired directions, thereby enabling the creation of non-line-of-sight (NLoS) paths when LoS links are blocked. By optimizing the IRS configuration, the effective channel between the VLC access point and the user can be improved even under adverse indoor conditions. Several studies have investigated IRS-assisted VLC architectures, focusing on beam steering and static element allocation to maximize the achievable data rate [5-8].

Despite recent advances, one critical aspect that remains underexplored is the impact of IRS-induced time delay. In IRS-assisted VLC systems, reflected signals may travel significantly longer paths than direct LoS components, resulting in dispersion that leads to frequency-selective fading. If not properly modeled, these effects can degrade system performance [9]. Recent work has emphasized the importance of incorporating delay effects into frequency-domain channel models for IRS-VLC systems [10], showing that reflection-induced delays are non-negligible and must be considered in system design. In recent years, Q-learning, a key method within reinforcement learning (RL), has attracted substantial attention in wireless communications due to its capacity to iteratively derive optimal action policies via Q-tables that encode state-action values. Its simplicity and effectiveness have led to successful implementations in several wireless network applications [11-13]. Deep reinforcement learning (DRL) has been increasingly adopted in IRS-assisted wireless systems, particularly to tackle non-convex optimization challenges. For example, Huang et al. [14] presented a DRL-based relay selection strategy in IRS-assisted cooperative networks, aiming to enhance throughput. Similarly, a multi-agent DRL approach was explored in [15], where the objective was to implement a buffer-aided relay selection mechanism within a secure cooperative IRS-enabled network.

1.1. Related Work

Further expanding on DRL’s applicability, the authors of [16] proposed an integrated machine learning framework combining LSTM-ARIMA position prediction and a dueling double deep Q-network (D³QN) algorithm to jointly optimize robot trajectories, RIS phase shifts, and power allocation in RIS-NOMA aided multi-robot indoor networks. The authors in [17] investigated mobile IRS elements mounted on autonomous robots in NOMA networks and employed federated learning approaches to optimize resource allocation and reflection configuration. In [18], the authors applied a recurrent neural network to dynamically optimize IRS placement in NLoS VLC environments, demonstrating significant SNR improvements using angle diversity receivers and adaptive learning strategies. A complementary effort in [19] leveraged a genetic algorithm-based solution to jointly optimize IRS placement and signal resource allocation in a NOMA-based VLC system, aiming to enhance link robustness. In addition, a DRL-based method was formulated in [20] to maximize secrecy capacity in IRS-assisted VLC environments, emphasizing physical-layer security. Lastly, the work in [21] introduced two iterative optimization techniques, the frozen variable method and a minorization-maximization algorithm, to incrementally improve spectral efficiency in multi-user VLC systems supported by IRS deployment. Finally, in line with the growing interest in the use of DRL for resource optimization in IRS-assisted VLC, the authors in [22] proposed a two-stage framework that jointly addresses user fairness and spectral efficiency. The proposed method begins with a Maximum Possible Fairness algorithm designed to balance resource allocation among users, followed by a DQL stage aimed at maximizing the overall spectral efficiency of the system. It is noted that applying DQL to jointly optimize IRS allocation and user-LED association under realistic VLC channel conditions, including time-delay effects, remains unexplored.

Recent research has also explored the joint optimization of LED transmission and IRS configuration in VLC systems. Liu et al. [23] investigated the maximization of the sum rate for NOMA-based VLC with intelligent optical reflecting surfaces, proposing joint optimization of IRS reflection coefficients and power allocation. Sun et al. [24] provided a comprehensive overview of optical IRS for VLC, introducing the association model for LED-IRS configurations and discussing beam alignment techniques. Liu et al. [25] proposed a robust layout optimization scheme using the non-dominated sorting genetic algorithm-II to jointly optimize access point positions and IRS array directions for blockage resistance. Zhao et al. [26] analyzed secrecy performance in IRS-aided VLC systems, optimizing LED-IRS element associations for physical layer security. An et al. [27] proposed a joint resource allocation framework for IRS-assisted VLC systems, optimizing optical IRS arrangement and power allocation to maximize energy efficiency. Cang et al. [28] investigated joint deployment and resource management for VLC-enabled RIS-assisted unmanned aerial vehicle (UAV) networks. However, these works assume flat-frequency channels and do not consider the time-delay effects introduced by differential propagation paths through IRS elements, which cause frequency-selective fading that significantly impacts achievable data rates.

Time dispersion and multipath propagation have been extensively studied in conventional VLC systems. Miramirkhani and Uysal [29] presented comprehensive indoor VLC channel models reporting root mean square (RMS) delay spreads of 10 -15 ns that cause inter-symbol interference (ISI) at high data rates. Tong et al. [30] investigated VLC channel characteristics in industrial IoT scenarios, modeling delay spread with distance-dependent statistical distributions and proposing link adaptation methods for multipath interference mitigation. Sejan et al. [31] provided a comprehensive survey on MIMO VLC, addressing ISI mitigation through spatial diversity and adaptive modulation techniques. To combat frequency-selective fading, T. Deepa et al. [32] proposed precoding matrix techniques for optical OFDM systems, while Zenhom et al. [33] evaluated various O-OFDM approaches for VLC systems, including DCO-OFDM, ACO-OFDM, and LACO-OFDM, proposing a Vandermonde-Like Matrix precoding technique to reduce the peak-to-average power ratio without degrading BER performance. Di Zheng et al. [34] introduced DCO-OTFS modulation to exploit delay-Doppler domain sparsity, achieving a 7 dB improvement over conventional OFDM in time-varying VLC channels. Farahneh H. et al. [35] analyzed adaptive OFDM in vehicular VLC networks with realistic multipath effects. While these works provide valuable insights into delay modeling and mitigation, they do not consider IRS-assisted scenarios where multiple reflected paths with distinct delays create complex frequency-selective channel responses.

From the above literature review, we identify a critical research gap: existing works have addressed joint LED-IRS optimization without considering time-delay effects, while others have addressed delay modeling in conventional VLC without IRS assistance. To the best of our knowledge, no prior work has jointly addressed IRS element allocation, LED-user association, and time-delay-induced frequency-selective fading within a unified optimization framework. Furthermore, learning-based approaches that can adapt to the complex, non-convex optimization landscape created by differential time delays remain unexplored. This paper bridges this gap by proposing a DQL framework that jointly optimizes IRS element allocation and LED-user association under a delay-aware, frequency-selective VLC channel model. A detailed numerical comparison against additional heuristic/RL baselines under the same delay-sensitive model is left for future work.

Against the given background, the motivations and contributions of this work are as follows:

1. We develop a theoretical framework to characterize the impact of IRS-induced time delay on the achievable rate using frequency-domain analysis and validate the proposed DQL-based optimization through extensive simulations.

2. We formulate a joint optimization problem (IRS allocation and LED association) that maximizes the system sum rate subject to per-user quality-of-service (QoS) constraints, taking into account IRS-induced time delay.

3. We develop a DQL-based algorithm to solve the joint optimization problem, enabling efficient IRS and user association learning.

The remainder of this paper is structured as follows: Section 3 describes the VLC system model, including channel formulation with LoS and IRS-assisted NLoS paths as well as time delay effects. Section 4 derives a closed-form expression for the sum rate using frequency-domain analysis. Section 5 presents the joint optimization problem for IRS element allocation and user association under QoS constraints. Section 6 introduces a DQL-based framework to solve the optimization problem efficiently in high-dimensional environments. Section 7 provides simulation results and discussion, highlighting the impact of time delay and device orientation. Finally, Section 8 concludes the paper and outlines potential directions for future research.

This work considers the downlink transmission in an IRS-assisted VLC system based on direct current optical frequency division multiplexing (DCO-OFDM). As shown in Figure 1, the system consists of L LEDs mounted on the ceiling and K randomly distributed users equipped with photodetectors (PDs) within the room. In addition, there are $N_{\mathrm{irs}}$ reflecting elements fixed on a wall of the room with a separation distance D between the elements. In IRS-assisted VLC, the transmitted signal reaches the receiver via two paths: LoS and NLoS, which are discussed in detail in the following subsections.

3.1. LoS Channel

The LoS channel gain between LED l and user k, denoted as $G_{\mathrm{LoS},k,l}$ , follows the Lambertian model [5]:

where A is the physical area of the PD, $d_{k,l}$ is the distance between the LED l and the user k, ${\varphi }_{k,l}$ and ${\psi }_{k,l}$ are the angles of irradiance and incidence respectively, ${\mathrm{\Psi }}_{\mathrm{FoV}}$ is the receiver field-of-view (FoV), ${𝔾}_f$ is the optical filter gain, and ${𝔾}_c$ is the optical concentrator gain defined as:

where $\xi$ is the refractive index. The Lambertian order m is:

where ${\mathrm{\Phi }}_{1/2}$ is the semi-angle at half the power of the LED.

3.2. IRS-Assisted NLoS Channel

The NLoS signal reflected via the IRS is modeled by considering a set of passive elements $N_{\mathrm{irs}}$ . Each IRS element reflects light from the LED l toward the user k by adjusting its orientation. The NLoS channel gain from LED l to user k through the IRS element n, denoted $G_{\mathrm{IRS},k,n,l}$ , is given by [5]:

where $\rho$ is the reflectivity of the IRS element, $d_{n,l}$ is the distance from LED l to IRS element n, $d_{k,n}$ is the distance from IRS element n to user k, and ${\varphi }_{n,l}$ , ${\psi }_{k,n}$ are the respective angles of irradiance and incidence. This work accounts for the time delay in light propagation for both LoS and NLoS paths. The overall channel impulse response (CIR) between user k and LED l is [9]:

where $\delta (·)$ denotes the Dirac delta function, ${\tau }_{k,l}=d_{k,l}/c$ is the LoS propagation delay, ${\tau }_{k,n,l}=(d_{n,l}+d_{k,n})/c$ is the IRS-reflected propagation delay via element n, and c is the speed of light. The channel frequency response (CFR), obtained as the Fourier transform of the CIR, is given by  [9]:

The signal-to-noise ratio (SNR) at frequency f is [9]:

where $T_s$ is the duration of the symbol, $E_k\left(f\right)$ is the power allocated to the user k at frequency f, $P_{\mathrm{opt}}$ is the optical power, $R_{\mathrm{pd}}$ is the PD responsivity, $\mathrm{\Gamma }$ is the SNR gap, $\kappa$ is the modulation scaling factor, and $N_0$ is the noise power spectral density. To investigate the effect of IRS-induced time delay across the frequency band, we expand the channel power response $|H_{k,l}{\left(f\right)|}^2$ as [9]:

For notational compactness in (8), we write $G_i\equiv G_{\mathrm{IRS},k,i,l}$ , $G_u\equiv G_{\mathrm{IRS},k,u,l}$ , ${\tau }_i\equiv {\tau }_{k,i,l}$ , and ${\tau }_u\equiv {\tau }_{k,u,l}$ , where ${\tau }_{k,i,l}=(d_{i,l}+d_{k,i})/c$ and ${\tau }_{k,u,l}=(d_{u,l}+d_{k,u})/c$ are the propagation delays via IRS elements i and u respectively. Here $G_{\mathrm{IRS},k,i,l}$ and $G_{\mathrm{IRS},k,u,l}$ are as defined in (4). The LoS delay ${\tau }_{k,l}=d_{k,l}/c$ is written explicitly in (8) without shorthand to avoid ambiguity. The full user- and LED-indexed notation is restored in Section 4. The four terms in (8) have the following physical interpretations: The first term $G_{\mathrm{LoS},k,l}^2$ is the direct LoS channel power. The second term captures the cross-gain between the LoS component and each IRS-reflected path, which gives rise to constructive or destructive interference depending on the differential delay $\Delta \tau ={\tau }_{k,i,l}-{\tau }_{k,l}$ . Note that (8) represents the full channel power response prior to IRS element assignment, summing contributions from all $N_{\mathrm{irs}}$ elements irrespective of user assignment. The user-specific rate expressions derived in Section 4 employ the binary indicator ${\widehat{s}}_{i,k}$ , defined in (10), to select only those IRS elements assigned to user k.

To analyze the impact of IRS-induced time delay in a frequency-selective VLC channel, we derive an expression for the achievable rate using DCO-OFDM under intensity modulation and direct detection (IM/DD). The rate for user k served by LED l is given by:

where $H_{k,l}\left(f\right)$ denotes the channel frequency response that includes both LoS and IRS-assisted NLoS components. We assume that the IRS element allocation is represented by an integer vector $\mathbf{s}={[s_1,s_2,\dots ,s_{N_{\mathrm{irs}}}]}^{\top }$ , where each $s_i\in \{1,2,\dots ,K\}$ indicates the user index to which the i-th IRS element is assigned. To incorporate this user-specific IRS mapping into the rate expression for user k, we define the binary indicator:

where ${\widehat{s}}_{i,k}=1$ if and only if the IRS element i is assigned to the user k, and zero otherwise. Note that by construction, ${\sum }_{k=1}^K{\widehat{s}}_{i,k}=1$ for all i, since each IRS element is assigned to exactly one user. By incorporating this binary selector into the channel model, the rate expression $R_{k,l}$ for the user k becomes:

where:

Here, the first term $G_{\mathrm{LoS},k,l}^2$ represents the direct LoS channel power. The second term ${\sum }_i{\widehat{s}}_{i,k}{G}_{i,k}^2$ captures the power of IRS-reflected paths assigned to user k. The third term represents the constructive/destructive cross-gain between the LoS and IRS-reflected components. The fourth term, with both selectors ${\widehat{s}}_{i,k}$ and ${\widehat{s}}_{u,k}$ , represents inter-element interference between pairs of IRS elements both assigned to user k; elements assigned to different users do not contribute to this term.

To separate the frequency-independent and frequency-dependent contributions, we apply the logarithmic identity:

where $A=\mathrm{\Lambda }(G_{\mathrm{LoS},k,l}^2,+,{\sum }_{i=1}^{N_{\mathrm{irs}}},{\widehat{s}}_{i,k},,G_{i,k}^2)$ collects all frequency-independent channel gain terms, and $B\left(f\right)=\mathrm{\Lambda }{N}_k\left(f\right)$ contains the frequency-dependent interference terms defined in (14). Applying (12) to Equation (11) yields:

where:

1. Approximation Strategy:

To simplify the integral in (13), we apply a cosine approximation. For small arguments, $\cos \left(x\right)\approx 1-{\displaystyle \frac{x^2}{2}}$ . However, to avoid inaccuracies for larger delays, we use a more balanced bound:

Applying this approximation to $N_k\left(f\right)$ in (14):

Substituting (16) into the second integral term of (13) and evaluating:

To account for approximation inaccuracies inherent in the cosine bound, a correction factor C is introduced. The final achievable rate approximation for user k served by LED l is:

This approximation provides analytical insight into how IRS-induced propagation delays affect the achievable rate in VLC systems operating under DCO-OFDM. The binary selectors ${\widehat{s}}_{i,k}$ and ${\widehat{s}}_{u,k}$ ensure that only IRS elements assigned to user k contribute to the rate computation, maintaining full consistency with the integer allocation vector $\mathbf{s}$ defined in constraint (20c).

The objective of this work is to jointly optimize IRS element allocation and user-LED association in an IRS-assisted VLC system while accounting for the time delays introduced by IRS reflections. Unlike most previous work, our formulation explicitly incorporates IRS-induced time delays into the system model, enabling more realistic and accurate performance optimization via DQL. The aim is to maximize the total system sum rate subject to minimum QoS requirements for all users.

5.1. Optimization Objective

The total system sum rate is defined as:

where $a_{k,l}\in \{0,1\}$ is the binary association variable indicating whether the user k is assigned to LED l, and $R_{k,l}$ is the achievable rate as defined in Equation (18). The optimization problem is formulated as:

Here, $R_{\mathrm{sum}}$ represents the total achievable sum rate of the system. The constraint (20b) ensures that each user meets the minimum data rate requirement $R_{\min }$ . The constraint (20c) ensures that each IRS element i is assigned to exactly one user via the allocation vector $\mathbf{s}$ . The corresponding binary selector ${\widehat{s}}_{i,k}$ , defined in (10), satisfies ${\sum }_{k=1}^K{\widehat{s}}_{i,k}=1$ for all i by construction and is used throughout the rate expressions to activate only the IRS elements belonging to user k. The constraint (20d) ensures that each user is served by exactly one LED.

The objective in (20a) to (20d) is to maximize the total sum rate $R_{\mathrm{sum}}$ using two discrete decision variables: the LED-to-user association $\mathbf{a}$ and the IRS element-to-user assignment $\mathbf{s}$ . Direct search is not practical when the system grows; therefore, we use a DQL to learn a policy that updates $\mathbf{a}$ and $\mathbf{s}$ to increase $R_{\mathrm{sum}}$ through repeated interaction with the environment, as summarized in Algorithm 1.

We model the problem as a Markov decision process (MDP) that is consistent with (20a) to (20d). The MDP observation ${\mathbf{o}}_t$ contains the current LED-user association, the IRS assignment, and the side information needed to evaluate the rates, such as user positions and channel summaries. The MDP action ${\alpha }_t$ is a single edit to the configuration, either reassigning an IRS element to a different user, or changing the LED of one user. Feasibility is enforced by construction: each user keeps exactly one serving LED, each IRS element is owned by exactly one user, and the QoS constraint $R_k\ge R_{\min }$ is handled by masking actions that would produce a next observation with any user below the threshold; if masking would remove all candidates, we keep the top actions by the predicted Q value to preserve exploration.

The reward is aligned exactly with the objective,

where $R_{k,l}$ is computed by using the delay-aware rate model defined earlier. No penalty terms are used; therefore, the agent learns to maximize the same function as in (20a).

The action-value function is approximated by a neural network $Q_{\theta }(\mathbf{o},\alpha )$ . We train it with a replay buffer and a target network. The temporal difference loss is:

where $\gamma$ is the discount factor and ${\theta }^{-}$ denotes the parameters of the target network. Exploration follows a $ϵ$ -greedy schedule that decays over time.

Table 1 specifies only the essentials of the function approximator; the input dimension follows the chosen observation encoding, and the output dimension follows the valid action set at each state. With these choices, the agent edits one part of the configuration at a time; feasibility is preserved at every step, and the learning signal equals the target objective; hence, the policy converges to configurations that give high $R_{\mathrm{sum}}$ under the constraints in (20b) to (20d). Training is performed offline for $E=3000$ episodes, and the learning curve in Figure 2 shows that the smoothed reward stabilizes after $\sim 1500$ episodes. In our MATLAB implementation, this offline training required approximately 4 h.

This section presents simulation results to evaluate the effectiveness of the proposed DQL framework for IRS-assisted VLC systems under realistic conditions, including IRS-induced time delay and user orientation variability. Our DQL-based method achieves performance trends similar to existing RL-based approaches [9,36], with added robustness under delay modeling and more complex IRS-user association dynamics. The simulation parameters are listed in Table 2. Users are randomly deployed within a $5\times 5\times 3$ m³ room. User device orientations are randomly sampled from a distribution representing realistic handheld usage. No dynamic blockages are explicitly modeled. The IRS is mounted on a vertical wall and assists communication by reflecting optical signals to users lacking a direct LoS path.

Figure 2 illustrates the evolution of the system sum rate during DQL training for a $30\times 30$ IRS configuration. In the early training stages, the agent explores suboptimal configurations due to random action selection. Over time, it learns effective IRS-user and LED-user assignments. The smoothed reward curve demonstrates stable convergence beyond 1500 episodes.

Figure 3 illustrates the accuracy of the derived analytical approximation for the achievable sum rate by comparing it to the result obtained via numerical integration of the full expression. In both cases, the system configuration (IRS-user and LED-user assignments) is determined using the proposed DQL algorithm. The close alignment between the two curves confirms that the analytical approximation closely matches the true rate computation, validating its use for performance evaluation and algorithm training.

Figure 4 highlights the importance of incorporating IRS-induced time delay into the system model. For each IRS size (10 × 10, 15 × 15, and 30 × 30 elements), two scenarios are compared: one models delay, and one assumes ideal synchrony. The results show that ignoring time delay leads to lower achievable rates. The performance gap increases with IRS size, as delay spread becomes more significant with longer reflected paths.

Figure 5 illustrates the impact of user device orientation on system performance under delay-aware modeling. Two scenarios are evaluated: fixed vs. random orientations across multiple IRS sizes ( $10\times 10$ , $15\times 15$ , and $30\times 30$ ). While both benefit from DQL-based optimization, fixed orientation achieves significantly higher rates due to improved alignment between users and IRS-reflected paths.

The results demonstrate that fixed orientation consistently outperforms random orientation, as predictable alignment enables the DQL agent to allocate IRS elements more effectively. The performance gap becomes more pronounced as the number of IRS elements increases, reflecting the growing sensitivity of larger IRS configurations to spatial misalignment caused by random device orientation, particularly in the presence of IRS-induced time delays. While both fixed and random orientation scenarios benefit from the proposed DQL optimization and delay-aware modeling, random orientation introduces greater variability in the channel response, which limits the achievable sum rate relative to fixed setups. Nevertheless, the results highlight the robustness of the DQL framework, which maintains strong performance even under the challenges posed by random user orientation. This underscores the importance of incorporating orientation dynamics into VLC system models and the need for adaptive learning-based control strategies capable of operating effectively under spatial uncertainty. The above analysis confirms the significance of user orientation in practical VLC systems and the need for learning-based adaptive IRS control that can tolerate spatial uncertainty while incorporating realistic delay effects.

7.1. Performance Under Different Channel Conditions

To evaluate robustness under practical propagation conditions, we compare three cases: (i) LoS-only, (ii) NLoS-only, and (iii) LoS+NLoS. Figure 6 shows the sum rate versus optical transmit power for $10\times 10$ elements. The combined LoS+NLoS case achieves the highest throughput across the full power range due to the additional IRS-reflected component. Importantly, the NLoS-only case maintains competitive performance relative to LoS-only, indicating that the IRS can help sustain connectivity under complete LoS blockage. Overall, these results demonstrate that the proposed DQL-based optimization remains effective across different channel conditions and is particularly beneficial in environments with intermittent or obstructed LoS.

This paper presented a DQL framework for joint optimization of IRS element allocation and user–LED association in IRS-assisted VLC systems. Unlike prior works, the proposed approach incorporates a frequency-selective channel model that includes IRS-induced time delays, which are important for realistic system-level modeling. A closed-form analytical approximation of the achievable rate under DCO-OFDM and IM/DD transmission was derived, capturing delay spread effects in the frequency domain. To address the resulting combinatorial optimization problem, a discrete-action DQL agent was employed to learn efficient IRS and LED configurations in dynamic indoor environments. Simulation results demonstrated that the analytical approximation closely matches the true achievable rate computed via numerical integration, supporting its use for system evaluation and training. Furthermore, incorporating IRS-induced delay into the system model improved performance compared to models that ignore such effects, especially for larger IRS arrays where delay spread becomes more significant. The results also highlighted the impact of user orientation, showing that fixed orientations yield higher performance than random ones.

Conceptualization, A.R.H. and H.A.; Methodology, A.R.H. and R.I.; Software, A.R.H.; Validation, A.R.H. and H.A.; Formal Analysis, A.R.H.; Investigation, A.R.H.; Resources, M.A.I. and H.A.; Data Curation, A.R.H.; Writing—Original Draft Preparation, A.R.H.; Writing—Review & Editing, A.Z., M.A.I. and H.A.; Visualization, A.R.H.; Supervision, A.Z., M.A.I. and H.A.; Project Administration, H.A. and M.A.I. All authors have read and agreed to the published version of the manuscript.

The simulation data supporting the findings of this study are available from the corresponding author upon reasonable request. Simulations were conducted using MATLAB (MathWorks, licensed through the University of Glasgow).

The content here.

The authors declare no conflicts of interest regarding this manuscript.

No external funding was received for this research.

The authors thank the James Watt School of Engineering, University of Glasgow, for providing the computational resources used in this work.

No AI-based tools were used during the preparation, language editing, or readability enhancement of this manuscript.