Context. In recent years, conflicting results have provided an uncertain view of the dust-attenuated star-forming properties of z greater than or similar to 4 galaxies. Aims. To solve this, we need to accurately measure the mean dust-attenuated properties of star-forming galaxies (SFGs) at 4 < z < 5 and therefore constrain the cosmic dust-attenuated star formation rate density (SFRD) of the Universe 1.3 Giga-years after the Big Bang. Methods. We used the deepest optical-to-near-infrared data publicly available in the Cosmic Evolution Survey (COSMOS) field to build a mass-complete (> 10(9.5) M-circle dot) sample of SFGs at 4 < z < 5. Then, we measured their mean dust-attenuated properties (i.e., infrared luminosity, < L-IR >; dust-attenuated star formation rate, < SFRIR >) by dividing our sample in three stellar mass (M-*) bins (i.e., 10(9.5) < M-*/M-circle dot < 10(10), 10(10) < M-*/M-circle dot < 10(10.5), and 10(10.5) < M-*/M-circle dot < 10(11.5)) and by stacking in the uv domain all archival Atacama Large Millimeter/submillimeter Array (ALMA) band 6 and 7 observations available for these galaxies. Then, we combined this information with their mean rest-frame ultraviolet (UV) emission measured from the COSMOS2020 catalog (i.e., UV luminosity, < L-UV >; UV spectral slope, ; and unattenuated SFR, < SFRUV >), and constrained the IRX (equivalent to L-IR/L-UV)-beta(UV), IRX-M-*, and SFR-M-* relations at z similar to 4.5. Finally, using these relations and the stellar mass function of SFGs at z similar to 4.5, we inferred the unattenuated and dust-attenuated SFRD at this epoch. Results. SFGs follow an IRX-beta(UV) relation that is consistent with that observed in local starbursts. Our measurements favors a steepening of the IRX-M-* relation at z similar to 4.5, compared to the redshift-independent IRX-M-* relation observed at z similar to 1 - 3. Our galaxies lie on a linear SFR-M-* relation, whose normalization varies by 0.3 dex, when we exclude or include from our stacks the ALMA primary targets (i.e., sources within 3 '' from the ALMA phase center). The cosmic SFRD( > M-*) converges at M-* less than or similar to 10(9) M-circle dot, with SFGs at 10(8) < M-*/M-circle dot < 10(9) contributing already less than 15% of the SFRD from all SFGs with M-* > 10(8) M-circle dot. The cosmic SFRD at z similar to 4.5 is dominated by SFGs with a stellar mass of 10(9.5 - 10.5) M-circle dot. Finally, the fraction of the cosmic SFRD that is attenuated by dust, SFRDIR(> M-*)/SFRD(> M-*), is 90 +/- 4% for M-* = 10(10) M-circle dot, 68 +/- 10% for M-* = 10(8.9) M-circle dot (i.e., 0.03 x M-star; M-star being the characteristic stellar mass of SFGs at this epoch) and this value converges to 60 +/- 10% for M-* = 10(8) M-circle dot. Conclusions. A non-evolving IRX-beta(UV) relation suggests that the grain properties (e.g., size distribution, composition) of dust in SFGs at z similar to 4.5 are similar to those in local starbursts. However, the mass and geometry of this dust result in lower attenuation in low-mass SFGs (less than or similar to 10(10) M-circle dot) at z similar to 4.5 than at z less than or similar to 3. Nevertheless, the fraction of the cosmic SFRD that is attenuated by dust remains significant (similar to 68 +/- 10%) even at such an early cosmic epoch.