Latest Post

(ML) MobileNetV2: Inverted Residuals and Linear Bottlenecks | by YEN HUNG CHENG | Jun, 2023

Photo by Bagus Hernawan on Unsplash在 MobileNetV2 中,作者使用了倒置殘差結構,其中快速連接位於狹窄的瓶頸層之間。他們採用輕量級深度卷積來過濾非線性源的特徵。同時,他們發現去除窄層中的非線性以保持表徵能力是非常重要的,並證明了這一改進可以提高性能。Depthwise Separable ConvolutionsLinear Bottlenecks簡單來說,Linear Bottlenecks 就是指在 MobileNetV1 最後做完 1x1 convolution 將激活函數 ReLU 拿掉,而為什麽這麼做,可以下方的 manifold of interest 介紹。manifold of interest在神經網絡中,每一層都會對輸入數據進行一些操作,比如卷積、激活、池化等等。這些操作產生的輸出值就被稱為該層的激活值。對於一個真實圖像的輸入集,每個層的激活值都形成了一個集合。我們可以把每個集合看作是一個流形(manifold)。這些流形反映了圖像的特徵,比如邊緣、紋理、形狀等等。長期以來,人們一直認為神經網絡中感興趣的流形可以嵌入到低維子空間中。MobileNetV1 成功利用,通過寬度乘數參數在計算和精度之間進行有效權衡。 按照這種直覺,寬度乘法器方法允許減少激活空間的維數,直到感興趣的流形跨越整個空間。然而,當我們回憶起深度卷積神經網絡實際上具有非線性的坐標變換(例如 ReLU)時,這種直覺就會失效。Examples of ReLU transformations of low-dimensional manifolds embedded in higher-dimensional spaces一開始在 2 維空間上建立一個 manifold of interest ,接下來會通過隨機矩陣 T 映射到 n 維空間後(manifold of interest 嵌入到更高維度的空間),接著進行 ReLU,最後再使用 T 的逆矩陣映射回原本的空間,也就是論文中所代表的圖。source在低(2、3)維度進行 ReLU 後,再映射回來原本的空間,可以發現原本螺旋的 manifold of interest 它被折疊了,並且其他部分訊息已經消失,反之可以發現在高(15、30)維度進行 ReLU 後,再映射回來原本的空間,雖然螺旋的 manifold of...

Read more

Trending

Popular

Latest Post

Unraveling the Design Pattern of Physics-Informed Neural Networks: Part 05 | by Shuai Guo | Jun, 2023

2.1 Problem 🎯In the application of Physics-Informed Neural Networks (PINNs), it comes as no surprise that the neural network hyperparameters, such as network depth, width, the choice of activation function, etc, all have significant impacts on the PINNs’ efficiency and accuracy.Naturally, people would resort to AutoML (more specifically, neural architecture...

Read more

Integrating AI into Your Finance Function

Emerging technologies such as artificial intelligence and machine learning have transformed the traditional finance function by making processes efficient, improving accuracy, and enabling data-driven decision-making. According to a Forrester survey, 98% of financial institutions believe that AI and ML can give them an edge and improve how they do business....

Read more
Page 1 of 138 1 2 138

Recommended