본문 바로가기
Programming 개발은 구글로/기타 정보

[WEB+AI] 32일차 딥러닝 GoogLeNet

by 40대직장인 2024. 11. 27.

 

딥러닝 CNN Models

 

강사: 이숙번님

 

  - numpy 이해

 

# 리스트
b1 = [1, 2, 3, 4, 5]
b2 = b1 * 2
print(b2)
r = [e * 2 for e in b1]
# b1 * 2의 결과가 모든 데이터에 2를 곱한 결과가 나오면 좋겠다.
print(r)

import numpy as np

a1 = np.array([1, 2, 3, 4, 5])
print(a1.shape)
print(a1)
print(a1 * 2)

a1 = np.array([1, 2, 3, 4, 5])
a2 = np.array([3, 4, 5, 6, 7])
print(a1 + a2)

C = np.array([1.5, 2.3, 3.5, 4.1, 5.5])
s = np.array([0.1, 0.45, 0.7, 0.0, 0.8])
print(C * s)

 

 

# 결과

[1, 2, 3, 4, 5, 1, 2, 3, 4, 5]
[2, 4, 6, 8, 10]
(5,)
[1 2 3 4 5]
[ 2  4  6  8 10]
[ 4  6  8 10 12]
[0.15  1.035 2.45  0.    4.4  ]

 

1. GoogLeNet

  • deep & wide
  • 22 layers but 5 million parameters
  • auxiliary classifier
  • inception module 사용
    • 1x1 convolution filter를 통한 파라미터 조절
    • channel-wise concat

 

  - GoogLeNet - Inception module

 

 

2. ResNet

  : ResNet은 딥러닝 모델이 깊어질수록 발생하는 기울기 소실(vanishing gradient) 문제를 해결하고, 매우 깊은 네트워크에서도 효율적으로 학습할 수 있도록 설계되었음.

 

  - skip connection & residual block

 

 

 

  • 100 layer 이상의 깊은 망
  • skip connection
  • residual learning building block
  • pre-activation 구조
  • 깊은 망 학습의 표준 모델이 됨.

 2.1. residual block

  - residual_block_v1

def residual_block_v1(n_filter, X):
    # X.shape (None, 32, 32, n_filter)

    H = tf.keras.layers.Conv2D(n_filter, kernel_size=3, padding="same")(X) # H.shape (None, 32, 32, n_filter)
    H = tf.keras.layers.BatchNormalization()(H)
    H = tf.keras.layers.Activation("relu")(H)

    H = tf.keras.layers.Conv2D(n_filter, kernel_size=3, padding="same")(H) # H.shape (None, 32, 32, n_filter)
    H = tf.keras.layers.BatchNormalization()(H)

    # skip connection
    H = H + X

    H = tf.keras.layers.Activation("relu")(H)

    return H

 

  - residual_block_v2

def residual_block_v2(n_filter, n_block, H):
    for i in range(n_block):
        H1 = tf.keras.layers.Conv2D(n_filter // 4, kernel_size=1, padding="same")(H)
        H1 = tf.keras.layers.BatchNormalization()(H1)
        H1 = tf.keras.layers.Activation("relu")(H1)
        H1 = tf.keras.layers.Conv2D(n_filter // 4, kernel_size=3, padding="same")(H)
        H1 = tf.keras.layers.BatchNormalization()(H1)
        H1 = tf.keras.layers.Activation("relu")(H1)        
        H1 = tf.keras.layers.Conv2D(n_filter, kernel_size=1, padding="same")(H1)
        H1 = tf.keras.layers.BatchNormalization()(H1)
        H = tf.keras.layers.Activation("relu")(H + H1) # skip connection
    return H

 

 

 

 

import tensorflow as tf

# 1. CIFAR-10 데이터 로드
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()
print(x_train.shape, y_train.shape)
print(x_test.shape, y_test.shape)

# 이미지 노멀라이즈 - 0~1 사이의 float으로 바꿈.
# 주의) 이걸 안해주면 결과가 제대로 안나옴.
x_train = x_train / 255.0
x_test = x_test / 255.0

# 원핫인코딩
y_train = tf.keras.utils.to_categorical(y_train)
y_test = tf.keras.utils.to_categorical(y_test)

print(x_train.shape, y_train.shape)
print(x_test.shape, y_test.shape)

 

X = tf.keras.Input(shape=[32, 32, 3])

H = tf.keras.layers.Conv2D(64, kernel_size=7, padding="same")(X)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(64, kernel_size=3, padding="same")(H)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.Conv2D(64, kernel_size=3, padding="same")(H)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(128, kernel_size=3, padding="same")(H)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.Conv2D(128, kernel_size=3, padding="same")(H)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(256, kernel_size=3, padding="same")(H)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.Conv2D(256, kernel_size=3, padding="same")(H)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(512, kernel_size=3, padding="same")(H)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.Conv2D(512, kernel_size=3, padding="same")(H)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Flatten()(H)
H = tf.keras.layers.Dense(1000, activation="relu")(H)
H = tf.keras.layers.BatchNormalization()(H)

Y = tf.keras.layers.Dense(10, activation="softmax")(H)

model = tf.keras.Model(X, Y)
model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])
model.summary()

 

 

  - skip connection 사용

X = tf.keras.Input(shape=[32, 32, 3])

H = tf.keras.layers.Conv2D(64, kernel_size=7, padding="same")(X)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.MaxPool2D()(H)
# (None, 16, 16, 64)

for i in range(3):
    H1 = tf.keras.layers.Conv2D(64, kernel_size=3, padding="same")(H)
    H1 = tf.keras.layers.BatchNormalization()(H1)
    H1 = tf.keras.layers.Activation("relu")(H1)
    H1 = tf.keras.layers.Conv2D(64, kernel_size=3, padding="same")(H1)
    H1 = tf.keras.layers.BatchNormalization()(H1)
    H = tf.keras.layers.Activation("relu")(H + H1) # skip connection
    # (None, 16, 16, 64)
H = tf.keras.layers.MaxPool2D()(H)
# (None, 8, 8, 64)

# 채널 수를 맞춰주는 1x1 Convolution
H = tf.keras.layers.Conv2D(128, kernel_size=1, padding="same")(H)
# (None, 8, 8, 128)

for i in range(4):
    H1 = tf.keras.layers.Conv2D(128, kernel_size=3, padding="same")(H)
    H1 = tf.keras.layers.BatchNormalization()(H1)
    H1 = tf.keras.layers.Activation("relu")(H1)
    H1 = tf.keras.layers.Conv2D(128, kernel_size=3, padding="same")(H1)
    H1 = tf.keras.layers.BatchNormalization()(H1)
    H = tf.keras.layers.Activation("relu")(H + H1) # skip connection
    # (None, 8, 8, 128)
H = tf.keras.layers.MaxPool2D()(H)
# (None, 4, 4, 128)

# 채널 수를 맞춰주는 1x1 Convolution
H = tf.keras.layers.Conv2D(256, kernel_size=1, padding="same")(H)
# (None, 4, 4, 256)

for i in range(6):
    H1 = tf.keras.layers.Conv2D(256, kernel_size=3, padding="same")(H)
    H1 = tf.keras.layers.BatchNormalization()(H1)
    H1 = tf.keras.layers.Activation("relu")(H1)
    H1 = tf.keras.layers.Conv2D(256, kernel_size=3, padding="same")(H1)
    H1 = tf.keras.layers.BatchNormalization()(H1)
    H = tf.keras.layers.Activation("relu")(H + H1) # skip connection
    # (None, 4, 4, 256)
H = tf.keras.layers.MaxPool2D()(H)
# (None, 2, 2, 256)

# 채널 수를 맞춰주는 1x1 Convolution
H = tf.keras.layers.Conv2D(512, kernel_size=1, padding="same")(H)
# (None, 2, 2, 512)

for i in range(3):
    H1 = tf.keras.layers.Conv2D(512, kernel_size=3, padding="same")(H)
    H1 = tf.keras.layers.BatchNormalization()(H1)
    H1 = tf.keras.layers.Activation("relu")(H1)
    H1 = tf.keras.layers.Conv2D(512, kernel_size=3, padding="same")(H1)
    H1 = tf.keras.layers.BatchNormalization()(H1)
    H = tf.keras.layers.Activation("relu")(H + H1) # skip connection
    # (None, 2, 2, 512)
H = tf.keras.layers.MaxPool2D()(H)
# (None, 1, 1, 512)

H = tf.keras.layers.Flatten()(H)
H = tf.keras.layers.Dropout(0.5)(H)
H = tf.keras.layers.Dense(1000, activation="relu")(H)
H = tf.keras.layers.BatchNormalization()(H)

Y = tf.keras.layers.Dense(10, activation="softmax")(H)

model = tf.keras.Model(X, Y)
model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])
model.summary()

 

 

  - residual_block 사용(v1) 34-layer

 

def residual_block_v1(n_filter, n_block, H):
    for i in range(n_block):
        H1 = tf.keras.layers.Conv2D(n_filter, kernel_size=3, padding="same")(H)
        H1 = tf.keras.layers.BatchNormalization()(H1)
        H1 = tf.keras.layers.Activation("relu")(H1)
        H1 = tf.keras.layers.Conv2D(n_filter, kernel_size=3, padding="same")(H1)
        H1 = tf.keras.layers.BatchNormalization()(H1)
        H = tf.keras.layers.Activation("relu")(H + H1)
    return H

 

X = tf.keras.Input(shape=[32, 32, 3])

H = tf.keras.layers.Conv2D(64, kernel_size=7, padding="same")(X)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.MaxPool2D()(H)

H = residual_block_v1(64, 3, H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(128, kernel_size=1, padding="same")(H)
H = residual_block_v1(128, 4, H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(256, kernel_size=1, padding="same")(H)
H = residual_block_v1(256, 6, H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(512, kernel_size=1, padding="same")(H)
H = residual_block_v1(512, 4, H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Flatten()(H)
H = tf.keras.layers.Dropout(0.5)(H)
H = tf.keras.layers.Dense(1000, activation="relu")(H)
H = tf.keras.layers.BatchNormalization()(H)

Y = tf.keras.layers.Dense(10, activation="softmax")(H)

model = tf.keras.Model(X, Y)
model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])
model.summary()

 

 

  - residual_block 사용(v2) 152-layer

 

def residual_block_v2(n_filter, n_block, H):
    for i in range(n_block):
        H1 = tf.keras.layers.Conv2D(n_filter // 4, kernel_size=1, padding="same")(H)
        H1 = tf.keras.layers.BatchNormalization()(H1)
        H1 = tf.keras.layers.Activation("relu")(H1)
        H1 = tf.keras.layers.Conv2D(n_filter // 4, kernel_size=3, padding="same")(H)
        H1 = tf.keras.layers.BatchNormalization()(H1)
        H1 = tf.keras.layers.Activation("relu")(H1)        
        H1 = tf.keras.layers.Conv2D(n_filter, kernel_size=1, padding="same")(H1)
        H1 = tf.keras.layers.BatchNormalization()(H1)
        H = tf.keras.layers.Activation("relu")(H + H1) # skip connection
    return H

 

X = tf.keras.Input(shape=[32, 32, 3])

H = tf.keras.layers.Conv2D(64, kernel_size=7, padding="same")(X)
H = tf.keras.layers.BatchNormalization()(H)
H = tf.keras.layers.Activation("relu")(H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(256, kernel_size=1, padding="same")(H)
H = residual_block_v2(256, 3, H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(512, kernel_size=1, padding="same")(H)
H = residual_block_v2(512, 8, H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(1024, kernel_size=1, padding="same")(H)
H = residual_block_v2(1024, 36, H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Conv2D(2048, kernel_size=1, padding="same")(H)
H = residual_block_v2(2048, 3, H)
H = tf.keras.layers.MaxPool2D()(H)

H = tf.keras.layers.Flatten()(H)
H = tf.keras.layers.Dropout(0.5)(H)
H = tf.keras.layers.Dense(1000, activation="relu")(H)
H = tf.keras.layers.BatchNormalization()(H)

Y = tf.keras.layers.Dense(10, activation="softmax")(H)

model = tf.keras.Model(X, Y)
model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])
model.summary()

 

 

3. LLM 직접 이용하기(with Hugging Face)

 

  - 허깅 페이스(Hugging Face)

  : 모델을 공유하는 사이트 Hugging Face.

  : https://huggingface.co/

 

  - 필요 라이브러리

  • transformers - 자연어처리 
  • diffusers - 이미지 생성
  • datasets - 데이터셋 
!pip install transformers

 

from transformers import pipeline

classfier = pipeline("sentiment-analysis")
results = classfier(["good", "soso", "not bad", "not good"])
print(results)

 

 

  - Pipeline(파이프라인) 예시

 

 

출처: AI Hub 교육과정 - WEB+AI (위 내용이 문제가 된다면 댓글에 남겨주세요. 바로 삭제조치하도록 하겠습니다.)

 

 

 

저작자표시

'Programming 개발은 구글로 > 기타 정보' 카테고리의 다른 글

[WEB+AI] 32일차 CNN Models  (0) 2024.11.26
[WEB+AI] 31일차 딥러닝 실습  (1) 2024.11.25
[WEB+AI] 30일차 딥러닝 + RNN  (0) 2024.11.22
[WEB+AI] 29일차 딥러닝 이미지 학습 및 LeNet5  (1) 2024.11.21
[WEB+AI] 28일차 딥러닝 2/2  (0) 2024.11.20

관련글

댓글

티스토리툴바