Ukuheha emibukeni yenethiwekhi ye-neural: ubuhle kanye neziyaluyalu

nimda March 25, 2025

0 0 8 minutes read

Ukuheha emibukeni yenethiwekhi ye-neural: ubuhle kanye neziyaluyalu

Isikhala sombuso wokwenza okubili kwe-neuron kusebenza ngokuhamba kwesikhathi silandela umkhangi.

Ingabe into eyodwa evamile phakathi kwezinkumbulo, ukusabela kwamakhemikhali ama-oscillating kanye nama-pendulum aphindwe kabili? Zonke lezi zinhlelo zine-bankin yokuheha izifundazwe ezingaba khona, njengezibuthe ezidonsela uhlelo ekubhekeni ama-trajectories athile. Izinhlelo eziyinkimbinkimbi ezinokufakwa okuningi zivame ukuvela ngokuhamba kwesikhathi, zikhiqiza izindlela eziyinkimbinkimbi futhi kwesinye isikhathi. Abakhangi bamele iphethini yokuziphatha yesikhathi eside yezinhlelo eziguqukayo – iphethini lapho uhlelo luguqulela khona ngokuhamba kwesikhathi nganoma yisiphi isikhathi izimo zalo zokuqala.

Amanethiwekhi we-Neural asuke e-Ubiquitous kwi-Era yethu yezobunhloli yokufakelwa, ngokuvamile esebenza njengamathuluzi anamandla okukhishwa nokuqashelwa kwephethini. Kodwa-ke, lezi zinhlelo zingabukwa futhi ngenye ilensi ekhangayo: njengoba izinhlelo ezinamandla ezivela futhi ziguqukele ekubeni yisifundazwe ngokuhamba kwesikhathi. Uma kusetshenziswane neziloops zamampendulo, ngisho namanethiwekhi we-neural alula angakhiqiza abakhangi abahle kakhulu, abasukela emijikelezweni emincane kuya ezinhlanganweni ezinesiphithiphithi.

Amanethiwekhi we-Neural njengezinhlelo ezinamandla

Ngenkathi amanethiwekhi we-neural ngomqondo ojwayelekile ajwayele ukwaziwa kakhulu ngemisebenzi yokukhipha, angabukwa futhi njengezinhlelo ezinamandla. Uhlelo olunamandla luchaza ukuthi amaphuzu atholakala kanjani endaweni yombuso avela ngokuhamba kwesikhathi ngokuya ngesethi yemithetho noma amabutho. Ngokwesimo samanethiwekhi we-neural, indawo yombuso iqukethe amaphethini wokusebenzisa ama-neurons, futhi umthetho wokuziphendukela kwemvelo unqunywa yizinsimbi zenethiwekhi, ukucwaninga, imisebenzi yokwenza kusebenze, namanye amaqhinga.

I-NNS yendabuko yenzelwe i-gradient feant ukuthola isiphetho sayo sokuhlangana. Kodwa-ke, lapho sethula impendulo – Ukuxhuma okuphumayo emuva kokufaka – inethiwekhi iba uhlelo olujwayelekile nohlobo oluhlukile lwamandla wesikhashana. Lezi zinqubo ezinamandla zingakhombisa izindlela ezahlukahlukene zokuziphatha, kusuka ekuguqulweni okulula kuya endaweni ehleliwe kumaphethini ayinkimbinkimbi axakile.

Ukuqonda Okuhehayo

Isikhangi seqoqo lezifundazwe lapho uhlelo luvame ukuvela kusuka ezinhlobonhlobo zezimo zokuqala. Lapho uhlelo lufinyelela okhangayo, luhlala luphakathi kwalelo kholothi lezifundazwe ngaphandle kokuthi lihlatshwe amandla ngamandla angaphandle. Abakhangi babambe iqhaza kakhulu ekwakheni izinkumbulo [1]ukusabela kwamakhemikhali kwe-oscillating [2]nezinye izinhlelo ezingaguquki ezinamandla.

Izinhlobo zabakhangi

Izinhlelo ezinamandla zingabonisa izinhlobo eziningana zabakhangi, ngayinye inezimpawu ezihlukile:

Khomba Ukuheha: Ifomu elilula kunazo zonke, lapho uhlelo luguqula iphuzu elilodwa elinqunyelwe kungakhathalekile ukuthi izimo zokuqalisa. Lokhu kufanekisela isimo esizinzile se-Equilibrium.
Khawulela imijikelezo: Uhlelo luhlala lube yi-orbit ephindaphindwayo, yakha i-loop evaliwe esikhaleni sesigaba. Lokhu kufanekisela ukusebenza kwe-oscillatory nesikhathi esinqunyelwe.
-Opopican (I-QuasiperoiodioIodiodioIodiodiodioiders: Uhlelo lulandela ama-trajectories avela ekwakhekeni okufana no-donut esikhaleni sesigaba. Ngokungafani nemijikelezo enomkhawulo, lezi zimbotshana azikaze ziphindaphinde ngempela kepha zihlala ziboshelwe esifundeni esithile.
Ethukwayo (Chaotic) okhangayo: Kubonakaliswa ngokuziphatha kwe-aperiodic okungaphindi kahle okwamanje kuhlale kufakwe ngaphakathi esifundeni esinqunyiwe sesigaba sesigaba. Abakhangi bakhombisa ukuncika okubucayi ezimweni zokuqala, lapho umehluko omncane uzokwethula imiphumela ebalulekile ngokuhamba kwesikhathi – uphawu lwesiphithiphithi. Cabanga umphumela uvemvane.

Ukubeka isikhathi

Esigabeni esilandelayo, sizongena sijule sibe yisibonelo sokwakhiwa okulula kakhulu kwe-NN OKUFANELE UKWENZIWA KAKHULU, futhi abonise izibonelo ezinhle. Sizothinta abathengisi beLhaapunov, futhi sihlinzeke ngokusetshenziswa kwalabo abafisa ukuzakhela ukukhiqiza i-Art Network Yobuciko be-neural Chex (hhayi emcabangweni we-AI akhiqiza).

Umdwebo 1. I-Nn Schematic and Ingxenye esizoyisebenzisela isizukulwane esihehayo. [all figures are created by the author, unless stated otherwise]

Sizosebenzisa i-NN eyodwa eyenziwe lula kakhulu nge-loop yempendulo. Ukwakhiwa kwakhiwa:

Ungqimba wokufaka:
- Uhlu lwesayizi d (lapha 16-32) okokufaka
- Sizokubhalela i-incronvendally them njenge-y₁, y₂, y₃, …, y_D ukugqamisa ukuthi lokhu kumephu kusuka kokuphuma
- Izenzo njengerejista yokushintsha egcina imiphumela edlule
Ungqimba olufihliwe:
- Iqukethe ama-neurons (lapha ambalwa kuno-d, ~ 4-8)
- Sizobabiza nge-X₁, X₂, …, x_Ni
- ithamo() Ukuvuselelwa kufakelwa ukuncipha
Ungqimba lokuphuma
- Ukukhishwa okukodwa kwe-Neuron (Y₀)
- Kuhlanganisa ukuphuma okufihliwe kwesendlalelo ngokuqothuka – ngokujwayelekile, sisebenzisa ukucwasana kokuphuma kwemiphumela ngokunehlisa; Lapha, sazisebenzisa ngokulinganisa, ngakho-ke ngokwezifiso zezisindo
Ukunye ukuxhumana:
- Okokufaka kufihliwe: isisindo matrix w[i,j] (iqalwa ngokungahleliwe phakathi -1 no-1)
- Kufihlwe ukuphuma: Isisindo se-Bias B[i] (iqalwa ngokungahleliwe phakathi kuka-0 no-s)
I-Foedback Loop:
- Ukukhishwa y y kudliswa emuva kungqimba lokufaka, ukudala imephu eshukumisayo
- Izenzo njengerejista ye-Shift (y₁ = edlule y₀, y₂ = eyedlule Y₁, njll.)
- Le mpendulo yikhona okudala indlela yokuziphatha kohlelo olunamandla
Amafomethi asemqoka:
- Ungqimba olufihliwe: u[i] = Σ (w[i,j] * y[j]); x[i] = ithamo(u[i]Isihlehlukene
- Okuphumayo: ₀ = σ (b[i] * x[i]Isihlehlukene

Izici ezibucayi ezenza le nethiwekhi iqede ukuheha:

Le khasi I-Foedback Loop Ijika inethiwekhi elula yokuphakelayo ibe uhlelo olunamandla
Le khasi Umsebenzi we-Nonlinear activation umsebenzi (ithamo) Inika amandla ukuziphatha okuyinkimbinkimbi
Le khasi Ukuqalwa kwesisindo okungahleliwe (Kulawulwa yimbewu engahleliwe) kwakha amaphethini ahlukene ezindeli
Le khasi ukukala isici s kuthinta amandla ashukumisayo wohlelo futhi kungakucindezela kube yimibuso yesiphithiphithi

Ukuze siphenye ukuthi uhlelo luyenzeka kanjani ku-chaos, sizobala ama-yapunov expentents ngamasethi ahlukene amapharamitha. ILYapunov Exponent yisilinganiso se- Ukungazinzisi kohlelo olunamandla…

[delta Z

[lambda = n_t sum_{k=0}^{n_t-1} ln frac{|Delta y_{k+1}|}{|Delta y_k|}]

… lapho n_tYinqwaba yezinyathelo zesikhathi, Δy_K Ibanga phakathi kwezifunda y (x_mina) no y (x_mina+ ε) ngesikhathi esifanele; I-ΔZ (0) imelela ukwahlukana kokuqala okungapheli (okuncane kakhulu) phakathi kwamaphoyinti amabili aseduze, futhi Δz

Asikhodi! Sizosebenzisa kuphela izitabane ze-numthan nezenzakalelayo zePython zokuqalisa.

import numpy as np
from typing import Tuple, List, Optional


class NeuralAttractor:
    """
    
    N : int
        Number of neurons in the hidden layer
    D : int
        Dimension of the input vector
    s : float
        Scaling factor for the output

    """
    
    def __init__(self, N: int = 4, D: int = 16, s: float = 0.75, seed: Optional[int] = 
None):
        self.N = N
        self.D = D
        self.s = s
        
        if seed is not None:
            np.random.seed(seed)
        
        # Initialize weights and biases
        self.w = 2.0 * np.random.random((N, D)) - 1.0  # Uniform in [-1, 1]
        self.b = s * np.random.random(N)  # Uniform in [0, s]
        
        # Initialize state vector structures
        self.x = np.zeros(N)  # Neuron states
        self.y = np.zeros(D)  # Input vector

Siqala i- NeuralAttractor Isigaba ngamanye amapharamitha ayisisekelo – Inani lama-neurons esingqimbeni esifihlekile, inani lezinto ezihlelweni zokufaka, ukukala isici sokuphuma, nembewu engahleliwe. Siyaqhubeka nokuqala izinsimbi nokucwaninga ngezikhathi ezithile, kanye no-x no-y. Lezi zinsimbi nokucwaninga ngeke zenziwe kahle – zizohlala zibekiwe, akukho kuvela i-gradient kulokhu.

    def reset(self, init_value: float = 0.001):
        """Reset the network state to initial conditions."""
        self.x = np.ones(self.N) * init_value
        self.y = np.zeros(self.D)
        
    def iterate(self) -> np.ndarray:
        """
        Perform one iteration of the network and return the neuron outputs.
        
        """
        # Calculate the output y0
        y0 = np.sum(self.b * self.x)
        
        # Shift the input vector
        self.y[1:] = self.y[:-1]
        self.y[0] = y0
        
        # Calculate the neuron inputs and apply activation fn
        for i in range(self.N):
            u = np.sum(self.w[i] * self.y)
            self.x[i] = np.tanh(u)
            
        return self.x.copy()

Okulandelayo, sizochaza umbono we-iTearation. Siqala yonke i-Iteration nge-Reportback Loop – Sisebenzisa umjikelezo werejista ye-Shift ngokushintsha zonke izinto ze-Y kwesokudla, futhi zihlanganise kakhulu y y₀ okuphumayo ukukubeka endaweni yokuqala yokufaka.

    def generate_trajectory(self, tmax: int, discard: int = 0) -> Tuple[np.ndarray, 
np.ndarray]:
        """
        Generate a trajectory of the states for tmax iterations.
        
        -----------
        tmax : int
            Total number of iterations
        discard : int
            Number of initial iterations to discard

        """
        self.reset()
        
        # Discard initial transient
        for _ in range(discard):
            self.iterate()
        
        x1_traj = np.zeros(tmax)
        x2_traj = np.zeros(tmax)
        
        for t in range(tmax):
            x = self.iterate()
            x1_traj
            x2_traj
            
        return x1_traj, x2_traj

Manje, sichaza umsebenzi ozothuthukisa imephu yethu yenethiwekhi ngaphezulu kwenombolo ye-TMAX yezinyathelo zesikhathi nokukhipha izifundazwe zezinzwa ezimbili ezifihliwe zokuthola ngeso lengqondo. Singasebenzisa noma yimaphi ama-neurons afihliwe, futhi singabona ngeso lengqondo isikhala sombuso we-3D, kepha sizonciphisa umcabango wethu ube yizilinganiso ezimbili.

Lona ungokolwazi lohlelo. Manje, sizochaza nje umugqa kanye nomlingo wengxenye ngombono omuhle.

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.collections as mcoll
import matplotlib.path as mpath
from typing import Tuple, Optional, Callable


def make_segments(x: np.ndarray, y: np.ndarray) -> np.ndarray:
    """
    Create list of line segments from x and y coordinates.
    
    -----------
    x : np.ndarray
        X coordinates
    y : np.ndarray
        Y coordinates

    """
    points = np.array([x, y]).T.reshape(-1, 1, 2)
    segments = np.concatenate([points[:-1], points[1:]], axis=1)
    return segments


def colorline(
    x: np.ndarray,
    y: np.ndarray,
    z: Optional[np.ndarray] = None,
    cmap = plt.get_cmap("jet"),
    norm = plt.Normalize(0.0, 1.0),
    linewidth: float = 1.0,
    alpha: float = 0.05,
    ax = None
):
    """
    Plot a colored line with coordinates x and y.
    
    -----------
    x : np.ndarray
        X coordinates
    y : np.ndarray
        Y coordinates

    """
    if ax is None:
        ax = plt.gca()
        
    if z is None:
        z = np.linspace(0.0, 1.0, len(x))
    
    segments = make_segments(x, y)
    lc = mcoll.LineCollection(
        segments, array=z, cmap=cmap, norm=norm, linewidth=linewidth, alpha=alpha
    )
    ax.add_collection(lc)
    
    return lc


def plot_attractor_trajectory(
    x: np.ndarray,
    y: np.ndarray,
    skip_value: int = 16,
    color_function: Optional[Callable] = None,
    cmap = plt.get_cmap("Spectral"),
    linewidth: float = 0.1,
    alpha: float = 0.1,
    figsize: Tuple[float, float] = (10, 10),
    interpolate_steps: int = 3,
    output_path: Optional[str] = None,
    dpi: int = 300,
    show: bool = True
):
    """
    Plot an attractor trajectory.
    
    Parameters:
    -----------
    x : np.ndarray
        X coordinates
    y : np.ndarray
        Y coordinates
    skip_value : int
        Number of points to skip for sparser plotting

    """
    fig, ax = plt.subplots(figsize=figsize)
    
    if interpolate_steps > 1:
        path = mpath.Path(np.column_stack([x, y]))
        verts = path.interpolated(steps=interpolate_steps).vertices
        x, y = verts[:, 0], verts[:, 1]
    
    x_plot = x[::skip_value]
    y_plot = y[::skip_value]
    
    if color_function is None:
        z = abs(np.sin(1.6 * y_plot + 0.4 * x_plot))
    else:
        z = color_function(x_plot, y_plot)
    
    colorline(x_plot, y_plot, z, cmap=cmap, linewidth=linewidth, alpha=alpha, ax=ax)
    
    ax.set_xlim(x.min(), x.max())
    ax.set_ylim(y.min(), y.max())
    
    ax.set_axis_off()
    ax.set_aspect('equal')
    
    plt.tight_layout()
    
    if output_path:
        fig.savefig(output_path, dpi=dpi, bbox_inches='tight')

    return fig

Imisebenzi ebhalwe ngenhla izothatha ama-trajectories esikhaleni esikhiqizwayo futhi ibone ngeso lengqondo. Ngoba isikhala sombuso singase sigcwaliswe kakhulu, sizokwenza Yeqa njalo nge-8, 16th noma i-32 yesikhathi ukuze uvikele ama-veector ethu. Futhi asifuni ukuhlela lokhu ngombala owodwa oqinile, ngakho-ke sikhona ukufaka umbala njengokwenza umsebenzi ngezikhathi ezithile (np.sin (1.6 * y_plot + 0.4 * x_plot)) Ngokusekelwe ku-X no-Y izixhumanisi ze-axis yesibalo. Abaphindaphindayo bezixhumanisi baphikisana futhi kwenzeka ukukhiqiza amamephu wombala omuhle, ngendlela oyithandayo.

N = 4
D = 32
s = 0.22
seed=174658140

tmax = 100000
discard = 1000

nn = NeuralAttractor(N, D, s, seed=seed)

# Generate trajectory
x1, x2 = nn.generate_trajectory(tmax, discard)

plot_attractor_trajectory(
    x1, x2,
    output_path='trajectory.png',
)

Ngemuva kokuchaza amapharamitha we-NN kanye ne-Iteration, singakwazi ukukhiqiza ama-trajectories esikhaleni sombuso. Uma sisebenzisa isikhathi esanele sihambahamba ngamapharamitha, sizothola okuthile okupholile (ngiyakwethembisa!). Uma ukusesha kwe-grid ye-grid ye-parameter ye-parameter ye-grid akuyona into yethu impela, singangezela umsebenzi ohlola ukuthi yini ingxenye yesikhala sombuso imbozwe ngokuhamba kwesikhathi. Uma ngemuva kwe-T = 100,000 iterations (ngaphandle kwezinyathelo zokuqala eziyi-1 000 “zokufudumeza”) Sithinte kuphela uhla oluncane lwendawo yesikhala sombuso, kungenzeka sibambeke endaweni. Lapho sesithole isikhangi esinganamahloni okuthatha isikhala esithe xaxa sombuso, singawuhlela ukusebenzisa amapharamu wokuhlela okuzenzakalelayo:

Umdwebo 2. Umkhawulo we-Cycle Chengeror.

Enye yezinhlobo ezizinzile zabakhangi yi- Khawulela umgibeli okhangayo (Amapharamitha: n = 4, d = 32, s = 0.22, imbewu = 174658140). Kubukeka njenge-trajectory eyodwa, evaliwe ye-loop endaweni yesigaba. I-orbit ilandela indlela ejwayelekile, yezikhathi ngezikhathi ezithile ngochungechunge lwesikhathi. Ngeke ngifake ikhodi yokubala okuthe xaxa lapha ukuze ugxile kwisici esibonakalayo sabakhangi abakhiqizwayo ngaphezulu, kepha umuntu angakuthola ngaphansi kwalesi sixhumanisi, uma unentshisekelo. I-LYAPONOV Exponent yalesi sikhangi (λ = -3.65) ayilungile, ekhombisa qinisa: ngokwezibalo, lokhu okuxoleli kuzoholela esimweni sokubola kwesistimu, noma ukuguqulwa, kulo mgodi wokuheha ngokuhamba kwesikhathi.

Uma siqhubeka nokwandisa isici sokukala, maningi amathuba okuthi sibhekane namagugu esekethelweni, futhi mhlawumbe kungenzeka sithole okuthile okuthokozisayo.

Nansi i I-Toroidal (Quasiperoiodiiodic) ekhangayo (Amapharamitha: n = 4, d = 32, s = 0.55, imbewu = 3160697950). Kusenesakhiwo esi-oda amashidi asongelayo emaphethini ahlelekile, athuthukile. I-LYAPONOV Exponent yalesi sikhangi inenani eliphakeme, kepha namanje akulungile (λ = -0.20).

Njengoba sithuthukisa isici sokulinganisa s, uhlelo luba luthambekele ngokwengeziwe ku-Chaos. Le khasi I-Drange (Chaotic) Chender Kuvela ngamapharamitha alandelayo: n = 4, d = 16, s = 1.4, imbewu = 174658140). Kuboniswa ngephethini engafanele, engalindeleki yezimpawu ezingakaze ziphinde. I-LYAPONOV Exponent yalesi sikhangi ihle (λ = 0.32), okukhombisa ukungazinzi (ukuphambuka kusuka esifundeni esiseduze kakhulu ngokuhamba kwesikhathi) nokuziphatha okuxakile. Lokhu kudonsela “umphumela wevemvane”.

Njengoba sithuthukisa isici sokulinganisa s, uhlelo luba luthambekele ngokwengeziwe ku-Chaos. Isikhangi esimangazayo (Chaotic) sivela ngamapharamitha alandelayo: n = 4, D = 16, s = 1.4, Imbewu = 174658140. Ibonakala ngephethini engekho emthethweni, engalindeleki ye-trajectories engasoze yaphinda. I-LYAPONOV Exponent yalesi sikhangi ihle (λ = 0.32), okukhombisa ukungazinzi (ukuphambuka kusuka esifundeni esiseduze kakhulu ngokuhamba kwesikhathi) nokuziphatha okuxakile. Lokhu kudonsela “umphumela wevemvane”.

Esinye nje isiqinisekiso sokuthi ama-aesthetics angaba yizibalo kakhulu, futhi okuphambene nalokho. Abakhangi ababukekayo kakhulu bavame ukuba khona emaphethelweni ezinxushu – cabanga ngakho okwesibili! Lezi zinhlaka ziyinkimbinkimbi ukukhombisa ukuziphatha eziyinkimbinkimbi, kepha kwayalwa ngokwanele ukuba kuhlale ndawonye ukuhambisana. Lokhu kusebenza ngokubona okuvela kumafomu e-ART ahlukahlukene, lapho ukulinganisela phakathi kwe-oda nokungapheli amandla kuvame ukudala okuhlangenwe nakho okubandakanya kakhulu.

Iwijethi esebenzayo ukukhiqiza futhi ibone ngeso lengqondo laba bahehe batholakala lapha. Ikhodi yomthombo iyatholakala, futhi, futhi imema eminye imininingwane. Imibono engemuva kwalo msebenzi yayiphefumulelwe kakhulu umsebenzi we-JC Story [3].