Connection weights determine an organizational topology for a network and allow units to send activation to each other. In a very rough sense, connection weights in ANNs can be regarded as analogous to neural synapses.
Input units code the problem being presented to the network. Output units code the network’s response to the input problem. Hidden units perform essential intermediate computations. Psychologically, the matrix of connection weights can be regarded as a network’s long-term memory. In cascade-correlation, there are cross-connections that bypass hidden units.