• <tr id="yyy80"></tr>
  • <sup id="yyy80"></sup>
  • <tfoot id="yyy80"><noscript id="yyy80"></noscript></tfoot>
  • 99热精品在线国产_美女午夜性视频免费_国产精品国产高清国产av_av欧美777_自拍偷自拍亚洲精品老妇_亚洲熟女精品中文字幕_www日本黄色视频网_国产精品野战在线观看 ?

    Perceptual Image Outpainting Assisted by Low-Level Feature Fusion and Multi-Patch Discriminator

    2022-08-23 02:18:02XiaojieLiYongpengRenHongpingRenCanghongShiXianZhangLutaoWangImranMumtazandXiWu
    Computers Materials&Continua 2022年6期

    Xiaojie Li,Yongpeng Ren,Hongping Ren,Canghong Shi,Xian Zhang,Lutao Wang,Imran Mumtaz and Xi Wu,

    1College of Computer Science,Chengdu University of Information Technology,Chengdu,610225,China

    2Xihua University,Chengdu,610039,China

    3University of Agriculture Faisalabad,Pakistan

    Abstract: Recently,deep learning-based image outpainting has made greatly notable improvements in computer vision field.However, due to the lack of fully extracting image information, the existing methods often generate unnatural and blurry outpainting results in most cases.To solve this issue,we propose a perceptual image outpainting method, which effectively takes the advantage of low-level feature fusion and multi-patch discriminator.Specifically, we first fuse the texture information in the low-level feature map of encoder,and simultaneously incorporate these aggregated features reusability with semantic (or structural) information of deep feature map such that we could utilize more sophisticated texture information to generate more authentic outpainting images.Then we also introduce a multi-patch discriminator to enhance the generated texture, which effectively judges the generated image from the different level features and concurrently impels our network to produce more natural and clearer outpainting results.Moreover, we further introduce perceptual loss and style loss to effectively improve the texture and style of outpainting images.Compared with the existing methods,our method could produce finer outpainting results.Experimental results on Places2 and Paris StreetView datasets illustrated the effectiveness of our method for image outpainting.

    Keywords: Deep learning; image outpainting; low-level feature fusion;multi-patch discriminator

    1 Introduction

    Nowadays,artificial intelligence(AI)has ushered in a new big data era.The improvement of AI also promotes deep learning technology to be widely employed in many fields[1–4],especially in image processing field.Combining deep learning technology with image processing method,AI system could acquire more available environmental information to make correct decisions.For example,applying deep learning-based image processing to the pattern recognition and automatic control field for the efficient analysis and real-time response,which is considered as a promising prospect.

    Recently, deep learning-based methods [5–8] have been widely applied to image inpainting task and have made remarkable achievements.Image inpainting,as a common image editing task,aims to restore damaged images and remove objects.Existing image inpainting methods can mainly be divided into two groups:non-learning methods and learning-based methods.The former group is composed of diffusion-based [9,10] and distribution-based approaches [11,12].Concretely, the diffusion-based approaches use the texture synthesis to fill the unknown parts, and search or collect the suitable pixels of known regions to diffuse into the unknown regions.These methods can generate meaningful textures for the missing regions.However, they generate inpainting results often with blurry and distorted contents when meet a big hole or sophisticated textures, because they fail to capture the semantic information of images.On the other hand, the distribution-based approaches utilize the whole dataset to obtain the data distribution information, and finally generate inpainting images.Similarly, due to only extracting low-level pixel information, they can’t produce a fine texture.By contrary,learning methods[13–15]generally use convolutional neural networks to extract the semantic information of images such that they could realize a natural,realistic and plausible inpainting result.

    Compared with image inpainting, image outpainting is studied relatively fewer in the image processing field.It uses the known parts of images to recursively extrapolate a complete picture.Moreover, image outpainting faces a greater challenge because of the less neighboring pixel information.Furthermore,the outpainting model must produce plausible contents and vivid textures for the missing regions.In practice,image outpainting can be applied in panorama synthesis,texture synthesis and so on.The generative adversarial network(GAN)[16,17]is commonly employed in image outpainting,and it is suitable for unsupervised learning on complicated distribution.GAN,as a generative model,aims to train jointly its generator and discriminator for an adversarial idea.Specifically,the generator minimizes the loss function,and the discriminator maximizes the loss function.Since the adversarial training promotes the generator to capture the real data distribution, the network can generate fine and reasonable images.

    Existing image outpainting methods generally fail to effectively extract image information(such as structure and texture information), resulting in the unclarity and unnaturalness of outpainting results.To generate more semantically reasonable and visually natural outpainting results,we present a perceptual image outpainting method assisted by low-level feature fusion and multi-patch discriminator(LM).It is known that the low-level features map with higher resolution could acquire plentiful detail information(such as location information and texture information).However,it contains less semantic information.The high-level feature map could acquire more semantic information, and it perceives the less detail information.Therefore, we first fuse the texture information in the lowlevel feature map of encoder, and simultaneously incorporate these aggregated features reusability with semantic (structural) information of deep feature map by element-wise adding such that we could utilize more sophisticated texture information to generate more authentic outpainting images.Moreover, we introduce a multi-patch discriminator to enhance the generated texture information and comprehensively judge the reality of outpainting images.We design its outputs as an×ntensor equal to judge the number of patches of an image,which could perceive the relatively bigger receptive field.Therefore,our multi-patch discriminator further effectively judges the generated image from the different level features and indirectly promotes the generator to grasp the real distribution of input data.This could impel our network to produce more natural and clearer outpainting images.

    Furthermore,we employ perceptual loss[18]to extract the high-level feature information of both generated images and ground truths.Therefore, our network could restrain the texture generation of outpainting regions.Meanwhile, style loss [19] is employed to estimate the relevance of different features extracted by pre-trained Visual Geometry Group 19(VGG19)network[20],and we further compute a Gram matrix to obtain the global style of outpainting images.In this way,our model can generate real and consistent outpainting results.

    In general,our contributions are as follows:

    (1) We effectively fuse and reuse the texture information of the low-level feature map of encoder and simultaneously incorporate these aggregated features reusability with semantic(structural)information of deep feature map in the decoder,which could utilize more sophisticated texture information to generate more authentic outpainting results with finer texture.

    (2) We propose two multi-patch discriminators to comprehensively judge the generated images from the different level features,which further enlarges receptive field of discriminator network and finally improves the clarity and naturalness of outpainting results.

    The rest part of paper is organized as follows: Section 2 presents related image outpainting works.The detail theory of our proposed method is illustrated in Section 3.Section 4 introduces our experimental results which include qualitative and quantitative comparisons with existing methods.In the last section,we present conclusions and future works.

    2 Related Work

    In the early time,image inpainting fills the missing areas through non-learning methods,including patch-based [21–23] and diffusion-based methods [24–26].Caspi et al.[27] use bidirectional spatial similarity to maintain the information of input data, which can be applied in retargeting or image inpainting.Nonetheless,the spatial similarity estimation costs a large number of computation resources.Barnes et al.[28]propose a PatchMatch method,using a fast nearest neighbor estimation to match reasonable patches.Therefore, PatchMatch could save expensive computation cost.These methods all assume that the missing contents come from the known regions,thus they search and copy the patches of known areas to fill unknown areas.By this way,they can generally produce meaningful contents for the missing regions.However,they often exhibit badly for complicated structures or bigger holes,due to they only gain low-level image information such as the non-learning statistics information and simple pixel information of images.

    Context Encoder (CE) [29] firstly applies the deep learning-based and GAN-based method to image inpainting task.It presents a new unsupervised learning method which is based on contextual pixel prediction.CE can be used to generate realistic contents according to known pixel information.Its overall network is an encoder-decoder architecture.The encoder maps the missing image into the latent space,and then the decoder utilizes these features of latent space to generate missing contents.A channel-wise fully-connected layer is introduced to connected encoder and decoder.In addition,both reconstruction loss and adversarial loss are used to train the CE model for realizing a sharp inpainting result.In this way, CE could simultaneously obtain both structure representation and semantic information of images.However,owing to the limitation of the fully-connected layer in the network,it fails to produce clear inpainting results.

    Chao et al.[30] propose a multi-scale neural patch synthesis algorithm, which is composed of content network and texture network.It can generate fine content and texture through training jointly the two networks.The content network is used to fill contents for the missing areas, while the texture network is used to further improve texture of output results generated by content network.Furthermore, in the texture network, a pre-trained VGG network is employed to force the patches in the inpainting regions to be perceptually similar to the patches in the known regions.Since they fully take the texture of missing regions into account, the network performs well for producing fine structures.However,due to the multi-scale learning which costs a lot of computation resources,this method has significant limitations.

    Then,Iizuka et al.[31]present a novel image inpainting method which guarantees the inpainting images with both local and global consistency.More specifically, it uses a local discriminator and a global discriminator to realize fine inpainting results.The local discriminator judges the inpainting areas to achieve local detail consistency, while the global discriminator judges the whole image to ensure the consistent overall structure.Thanks to ensuring the consistency of local and global details,the model could produce much finer inpainting results.Moreover, it also achieves a more flexible inpainting without the limitation of image resolution and missing shape.

    To get over the influence of subordinate pixels in the missing regions,Liu et al.[32]create a partial convolution for irregular image inpainting.In the method, they use the masked and renormalized convolution to force the network to focus on the valid pixels of input images.Moreover, they also present a method to automatically update the mask value for the next convolutional layer.By this way,the influence of subordinate information can be reduced in some degree,which promotes the network to process the input image more effectively.Ultimately,they realize natural and clear inpainting results.

    Zheng et al.[33] propose a pluralistic image inpainting method (PICnet), which could produce multiple output for one input image.The most image inpainting methods only output one result,due to the limitation of one instance label provided by the ground truth.To let the model output diverse inpainting results, they invent a novel probabilistic theory to settle the problem.In addition, their network architecture contains two parallel paths,which are composed of the reconstructive path and the generated path.Concretely,the reconstructive path is used to obtain the distribution information of missing regions, and finally reconstructing a complete image.On the other hand, the generated path utilizes the distribution information of reconstructive path to guide the generation of missing images.By sampling from the variational auto-encoder(VAE)(another generative model),the network can produce pluralistic inpainting images.Owing to the considering of prior distribution of missing regions,they not only generate high-quality results but also create the diversity of images.

    Mark et al.[34] recently apply GAN to the image outpainting for painting outside the box(IOGnet).They employ the deep learning-based GAN approach to outpaint the panorama contents for the sides of missing images,and finally recursively expand the parts beyond the border.Furthermore,they adopt a three-stage strategy to stabilize the training process.In the first stage,the generator is trained by the L2 distance between the generated images and the ground truths.In the second stage,the discriminator is trained alone according to the adversarial loss.In the last stage,the generator and discriminator are trained jointly through the adversarial loss.Finally,the model could even generate a five-time outpainting result than the original input.However, the obscure contents appear in the outpainting parts.As a result,the work needs to be improved in some aspects.

    3 Perceptual Image Outpainting Assisted by Low-Level Feature Fusion and Multi-Patch Discriminator(LM)

    To produce high-quality outpainting results, we present a simple perceptual image outpainting method assisted by low-level feature fusion and multi-patch discriminator.Moreover, we simultaneously employ both perceptual loss and style loss to improve the texture and style of outpainting images.Network architecture will be introduced in Subsection 3.1, and the rest of subsections are used to introduce the principle of our method.

    3.1 Network Architecture

    As shown in Fig.1, a simple GAN-based network, mainly consisting of the generator and discriminator, is used in our network.Firstly, our encoder in generator maps input images (bothImandIc) into a latent feature space.We first fuse the texture information in the low-level feature map of encoder, and simultaneously incorporate these aggregated features reusability with semantic(structural) information of deep feature map by element-wise adding in decoder.This could utilize more sophisticated texture information to generate more authentic outpainting images.Furthermore,the inference module(yellow block)connects encoder with decoder for utilizing the latent feature more effectively.In fact,the inference module is equal to the function of VAE[35],which computes the mean and variance of latent features to sample useful features.Finally,to generate more realistic results,we inject outpainting image into the pre-trained VGG[36]network for obtaining the feature information,which will be used to compute perceptual loss and style loss.In addition, we use the Least Squares Generative Adversarial Network(LSGAN)loss[37]to stabilize the training of our model.Then we present a multi-patch discriminator to enhance the generated texture information, which effectively judges the generated image from the different level features and impels our network to produce more natural and clearer outpainting images.

    Figure 1:Overview of our network architecture

    3.1.1 Generator

    Fig.1 shows that our network structure consists of two paths:yellow path in the top and blue path in the bottom.Note that the former path aims to reconstruct inpainting images and the latter path aims to generate outpainting results.In the training, both masked imagesImandIc(complement ofIm)are concatenated by the channel-wise operation such that both can be simultaneously processed.Then we detach output features into both different inference modules(yellow block)to compute their latent features’mean and variance,which will be used to sample latent features.To simultaneously deal with both latent features,we concatenate both sampling features and feed them into the decoder.To easily grasp more sophisticated texture information and generate more authentic outpainting images,when decoder processes the latent features we fuse the texture information in the low-level feature map of encoder, and simultaneously incorporate these aggregated features reusability with semantic(structural) information of deep feature map by element-wise adding in decoder.It is formally defined as:

    whereFiis i-th layer’s aggregated features,Idenotes input image,Eiis i-th layer in the encoder, ⊕denotes channel-wise concatenation, andCDis down-sampling operation.Namely, we first downsample (i-1)-th layer’s features, and concatenate the down-sampling features with the i-th layer’s features by channel-wise concatenation.Therefore,Ficontains (i-1)-th and i-th layers’ aggregated feature information (see Eq.(1)).Then, we could pass aggregated featuresFiinto the decoder via element-wise adding.Therefore, the network could generate more sophisticated texture for the generated images.Finally,we produce both reconstructive imageIrecand generated imageIgen.

    3.1.2 Discriminator

    We design multi-patch discriminators (both Discriminator 1 and Discriminator 2) to enhance the generated texture information, which effectively judges the generated imageIgenandIrecfrom the different level features and impels our network to produce more natural and clearer outpainting images.Formally,it is defined as:

    whereis the generator’s adversarial loss,Diis the i-th layer of discriminator,andIgenis the generated image.Specifically, we judge the output patches in the last three layers of discriminator are real or fake.From the multi-patch information, the discriminator could effectively reinforce the ability of judgement for the output patch of discriminator (see Eq.(2)).Therefore, the discriminator could comprehensively judge an input image is real or fake.Finally,the real distribution of data is grasped by the generator,and the model could produce finer outpainting results.

    3.2 Perceptual Loss and Style Loss

    To further improve the texture and style of outpainting images and generate more realistic result,we simultaneously introduce both perceptual loss and style loss.Perceptual loss aims to extract semantic(structure)feature information via the pre-trained VGG19 network.By constraining theL1distance of these features,it can force outpainting results perceptually close to ground truths.Formally,the perceptual loss is defined as:

    whereIgenis the generated image andIgtis the ground truth.Φi(·)denotes the i-th layer features map of VGG.Actually, the perceptual loss is used to measure the difference of corresponding features extracted by VGG.The features in the convolutional neural network generally represent the semantic information of images such as the low-level textures or high-level attributes.Through penalizing these features dissimilar to the feature labels in the VGG, the outpainting parts can be improved in some degree.Thanks to the applying of perceptual loss in the training of GANs, the generator could be gradually tuned to produce a finer output result.

    Style loss aims to extract the general style of generated images and ground truth.Concretely,to capture the overall style,we calculate the Gram matrix of their features extracted by VGG network.As a result of theL1norm constraint on the corresponding Gram matrices, the outpainting images will approach the realistic style by degrees.Analogously,the style loss is defined as follow:

    where GΦi(·)denotes the Gram matrix of i-th layer’s feature extracted by VGG network.In fact,Gram matrix is the covariance matrix of eigenvectors in the Euclidean space,and it estimates the correlation of pair eigenvectors.Convolutional Neural Network(CNN)extracts the low-level texture information of images in the shallow layer,while in the deeper layer it obtains the high-level semantic information.The genuine attribute of an image is up to the combination of low-level and high-level information.Therefore, it can be used to measure the correlation of different features, including the important essence of images.Since we force the style of outpainting images to be similar to the style of ground truths,our model can produce the outpainting results with natural and authentic appearance.

    3.3 Other Loss

    Moreover,we apply the loss from PIC.Formally,

    where the subscriptrdenotes the reconstructive path (see yellow path in Fig.1), andgdenotes the generated path (see blue path in Fig.1).LKLis the KL loss for restraining the distribution of both reconstructive images and generated images.Lappis the reconstruction loss,andLadis the adversarial loss for GAN.

    In our model,the total loss is defined as follow:

    whereλ1=0.1,λ2=250.0 in our experiments.

    4 Experimental Results

    4.1 Dataset

    We evaluate our method on both Places2[38]and Paris StreetView[39]datasets.Places2 dataset is a natural scene dataset which is widely used in image outpainting.We divided Places2 into training set 308,500 and test set 20,000.Paris dataset is building view dataset,and we divided Paris into training set 14,900 and test set 100.All of images are resized to 128 × 128 and normalized to [0,1].These normalized inputs of[0,1]can accelerate the training of model,and it also summarizes the statistical distribution of uniform samples.

    4.2 Experimental Setup

    All experiments are implemented on Pytorch framework with Ubuntu 16.04, Python 3.6.9,PyTorch 1.2.0,and RTX 2080TI GPU.Moreover,we set a batch size of 64,and use Adam optimizer to train our network with an initial learning-rate of 0.00001, and the orthogonal method is used to initialize the parameters of model.Although the network consists of two paths,it is trained in an endto-end style.We also employ a LSGAN loss to make the training stable.In the training procedure,we update the discriminator once and update the generator once to complete the adversarial training.The test input is the masked image with missing center regular holes or long strips.Note that,during test,we only use the bottom blue path to output final results.During training time,our model spent 6 days and 5 days on Places2 and Paris datasets respectively, while PICnet spent 7 days and 6 days on Places2 and Paris datasets respectively.Therefore,it proves that our method is more efficient for training times.

    4.3 Evaluation Metrics

    We compare our method(PICnet-SP-LM)with PICnet and its variants(PICnet-S(PICnet with style loss) and PICnet-SP (PICnet with style loss and perceptual loss)) in terms of qualitative and quantitative aspects.In the qualitative aspect,we can visually judge whether the outpainting parts are fine or bad.In the quantitative aspect, six types of metrics are used to measure the performance of different methods:

    (1) Inception Score(IS)[40]is a common quantitative metric which is used to judge the quality of generated images.GANs,which can generate clear and diverse images,are considered as good generated models.IScan be used to measure the clarity and diversity of images.Formally,ISis defined as follow:

    wheregis the generator,ydenotes the generated image,andzis the label predicted by the pre-trained Inception V3 model.The higher IS score signifies that the generated images are clearer and more diverse.

    (2) Another metric usually used to measure the quality of GAN is Frechet Inception Distance(FID)[41].FID aims to estimate the distance between the feature vectors of generated image and ground truth in a same domain.Formally:

    wherexdenotes the ground truth andydenotes the generated image.μis the mean value of eigenvectors, andΣis the covariance matrix of eigenvectors.The lower FID score also means that the generated images are higher-quality for clarity and diversity.

    (3) Structural similarity (SSIM) aims to evaluate the quality of image based on the luminance,contract and structure of two images.Formally,

    wherex,ydenote ground truth and generated image respectively,μxis the mean value ofx,σxdenotes the variance ofx,andσxydenotes the covariance ofxandy.The higher SSIM means the generated images possess finer luminance,contract and structure.

    (4) Peak signal-to-noise ratio(PSNR)is a full reference estimation metric,and it is used to measure the degree of image distortion.Formally,

    wheredenotes the max pixel value in an image,andMSEis the abbreviation of mean square error.A higher PSNR score signifies the generated images are more natural.

    (5)L1loss measures the pixel-wise difference by computing the L1 distance.Formally,

    wherex,ydenote ground truth and generated image respectively, (i, j) denotes the position in the image, andmsignifies the number of total elements.The lowerL1loss means generated images are closer to ground truths for pixel-wise difference.

    (6) RMSE is used to measure the deviation between generated image and ground truth.Formally,

    wherex,ydenote ground truth and generated image respectively,(i,j)denotes the position in the image,andmsignifies the number of total elements.Similarly,the lower RMSE means generated images are closer to ground truths.

    4.4 Qualitative Results

    Figs.2 and 3 illustrate the qualitative results of different methods with 64 × 64 valid pixels input on the different datasets.It is easy to see that the original PICnet generated blurry textures and distorted structures in the outpainting areas (see Fig.2c).To solve the existing problems, we first introduce perceptual loss and style loss.For style loss, PICnet-S (PICnet with style loss) could improve the existing distorted structures,and these coarse results become much smoother(see Fig.2d).Furthermore,we used both style loss and perceptual loss in PICnet(denoted as PICnet-SP)(PICnet with style loss and perceptual loss) to improve the outpainting results.We can see the details from Fig.2e.Compared with the results of PICnet-S,PICnet-SP exhibits better on the Places2.For instance,with style loss and perceptual loss,the results are more realistic and more natural in general.To further improve the quality of outpainting images, we fuse the texture information in the low-level feature map of encoder, and simultaneously incorporate these aggregated features reusability with semantic(structural)information of deep feature map by element-wise adding in decoder.Simultaneously,we designed multi-patch discriminator into the network.This could utilize more sophisticated texture information to generate more authentic outpainting images(see Fig.2f).We can see that our PICnet-SP-LM achieved a more authentic outpainting result.Moreover,we also find a similar effect on the Paris dataset.In the Fig.3c, the vanilla PICnet method produces poor results which are filled with fuzzy contents and shadows.However, the outpainting parts are improved a lot when we add style loss alone or both style loss and perceptual loss (see Figs.3d and 3e).Specifically, these shadows disappear in some degree and the blurry textures become clearer.Fig.3f with the low-level feature fusion and the multi-patch discriminator exhibits better than the former methods.This proves that low-level feature fusion and multi-patch discriminator could promote the network to generate higherquality outpainting images.

    Figure 2:Qualitative results of different methods with 64×64 valid pixels’input on the Places2 dataset

    To further evaluate the effectiveness of our method, we set 128 × 64 valid pixels as the input of network (see Figs.4b and 5b).Figs.4 and 5 show the qualitative results of different methods on Places2 and Paris,respectively.From Figs.4c and 5c,the original PICnet produces poor outpainting results with apparent boundaries and warped structures.Nonetheless, these situations are greatly improved when we use perceptual loss and style loss.In the Figs.4 and 5,the structures become more natural and clearer(see Figs.4d,4e, 5d and 5e).Moreover,Figs.4f and 5f,generated by our PICnet-SP-LM,reach the higher effect than the others.Thus,these results once again demonstrate that both low-level feature fusion and the multi-patch discriminator are instrumental for network to improve the quality of outpainting images.

    Figure 3: Qualitative results of different methods with 64 × 64 valid pixels’ input on the Paris StreetView dataset

    Figure 4: Qualitative results of different methods with 128 × 64 valid pixels’ input on the Places2 dataset

    Figure 5: Qualitative results of different methods with 128 × 64 valid pixels’ input on the Paris StreetView dataset

    4.5 Quantitative Results

    The qualitative results of different methods on both Paris and Places2 datasets with different inputs are shown in Tabs.1–4.The quantitative results with 64×64 valid pixels’input on Paris and Places2 are shown in the Tabs.1 and 2.In the Tab.1,we exhibit the quantitative metrics of 20,000 test images on the Places2.In the experiments, our method with low-level feature fusion and the multipatch discriminator also achieves better metrics.Specially, our PICnet-SP-LM method achives the lower 30.81 for FID, signifying our model can realize clearer and more diverse outpainting results.The higher PSNR of 13.72 and SSIM of 0.4261, proving our results have a better image structure.Besides, we also obtain lower L1 loss of 34.47 and RMSE of 64.76, which indicates our results are closer to ground truths for pixel difference.Tab.2 shows the quantitative metrics on the Paris.As result of the limitation of the 100 test images of Paris,we only measure the metrics SSIM and RMSE.From the quantitative results, low-level feature fusion and multi-patch discriminator again improve the results generated by the vanilla PICnet.

    Furthermore, Tabs.3 and 4 show the quantitative results of different methods with 128 × 64 valid pixels’ input on Places2 and Paris.The effect of low-level feature fusion and the multi-patch discriminator once presents in the tables.Vanilla PICnet method produces the poor results which have lower-quality quantitative metrics.Contrarily,the quantitative metrics of outpainting results produced by PICnet-SP-LM can realize a better degree.Specially,with the effect of the low-level feature fusion and the multi-patch discriminator,PICnet-SP-LM achieves higher PSNR of 16.78 and SSIM of 0.6452 on the Places2 dataset.Meanwhile,PICnet-SP-LM also realizes the lower FID of 9.99 and L1 loss of 19.25.In addition,on the Paris dataset,PICnet-SP-LM also exhibits better for SSIM and RMSE.All the experiments demonstrate that both low-level feature fusion and the multi-patch discriminator are beneficial for outpainting network to improve the quality of outpainting images.

    Table 1:Quantitative results of different methods with 64×64 valid pixels’input on the Places2 dataset

    Table 2:Quantitative results of different methods with 64×64 valid pixels’input on Paris StreetView.Because the limitation of the 100 test images of Paris StreetView,we only evaluate the SSIM and RMSE

    Table 3: Quantitative results of different methods with 128 × 64 valid pixels’ input on the Places2 dataset

    Table 4:Quantitative results of different methods with 128×64 valid pixels’input on Paris StreetView.Because the limitation of the 100 test images of Paris StreetView,we only evaluate the SSIM and RMSE

    4.6 Ablation Study

    In addition,we also implement other experiments for further selecting the better PICnet-SP-LM method.Tab.5 is the quantitative results of implemental experiments on the Places2 dataset.Specifically, PICnet-SP-LM-1 and PICnet-SP-LM-2 are the different hyper parameters for reconstruction loss and KL loss,respectively.(PICnet-SP-LM-1 with hyper parameter 20 for reconstruction loss and hyper parameter 20 for KL loss, and PICnet-SP-LM-2 with hyper parameter 20 for reconstruction loss and hyper parameter 40 for KL loss.)From the experimental results,PICnet-SP-LM-1 achieves a better degree.Thus,PICnet-SP-LM-3 and PICnet-SP-LM-4 adopt the hyper parameters of PICnet-SP-LM-1.PICnet-SP-LM-3 utilizes one layer’s aggregated features,and PICnet-SP-LM-4 utilizes two layers’aggregated features.Apparently,PICnet-SP-LM-4 utilizing more aggregated features achieves a better effect.Therefore, PICnet-SP-LM-4 is an optimal experimental setup, which could generate more natural and more realistic outpainting results.Moreover, for the qualitative aspect, the results generated by PICnet-SP-LM-4 are also clearer and more authentic than other methods.In the Fig.6,we also select some outpainting results with borders in baseline model.Then we relieve or eliminate these borders through gradually adding our core blocks, which could present the obvious effect of these core blocks.

    Table 5: Quantitative results of ablation study with 64×64 valid pixels’input on the Places2 dataset

    Figure 6:Qualitative results of ablation study on the Places2 dataset.(a)Input,(b)PICnet-SP-LM-1,(c)PICnet-SP-LM-2,(d)PICnet-SP-LM-3,(e)PICnet-SP-LM-4

    5 Conclusion

    In fact,image outpainting plays an important role in image processing field,and it can be also used to promote the image inpainting.In this paper, we present a perceptual image outpainting method,which is assisted by low-level feature fusion and multi-patch discriminator.In details, we first fuse the low-level texture information in the encoder,and simultaneously incorporate these fused features with semantic(or structural)information of deep feature map,which could promote the network to generate finer outpainting results.At the same time, we also present a multi-patch discriminator to enhance the generated image texture,which effectively judges the generated image from the different level features and impels our network to produce more natural and clearer outpainting results.To fully evaluate our model,we implement experiments on Places2 and Paris dataset.Finally,the experimental results show that our method is better than PICnet for qualitative effects and quantitative metrics,which proves the effectiveness and efficiency of our method for image outpainting task.In the future,we will further study more challenging image outpainting field,such as the input images with bigger missing regions.We also try to realize higher-quality outpainting results.

    Acknowledgement:I would like to thank those who helped me generously in this research.

    Funding Statement:This work was supported by the Sichuan Science and Technology program(2019JDJQ0002, 2019YFG0496, 2021016, 2020JDTD0020), and partially supported by National Science Foundation of China 42075142.

    Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.

    大香蕉久久成人网| 欧美中文综合在线视频| 热re99久久国产66热| 国产成人aa在线观看| 日韩制服丝袜自拍偷拍| 欧美日本中文国产一区发布| 日本午夜av视频| 国产激情久久老熟女| 色播在线永久视频| 亚洲精品一区蜜桃| 久久女婷五月综合色啪小说| 少妇人妻 视频| 天美传媒精品一区二区| 亚洲成人一二三区av| 久久毛片免费看一区二区三区| 成年美女黄网站色视频大全免费| 人人妻人人澡人人看| 综合色丁香网| 999久久久国产精品视频| 亚洲国产精品999| 男女啪啪激烈高潮av片| 亚洲av电影在线观看一区二区三区| 亚洲精品国产av成人精品| 久久人人97超碰香蕉20202| 十八禁网站网址无遮挡| 久久精品国产自在天天线| 久久久久久久久免费视频了| 黄色 视频免费看| 欧美激情 高清一区二区三区| 天天躁夜夜躁狠狠久久av| 777米奇影视久久| 久久精品国产综合久久久| 久久久久久久国产电影| 亚洲成色77777| 日韩伦理黄色片| 日韩成人av中文字幕在线观看| 久久人人爽人人片av| 久久久a久久爽久久v久久| 欧美最新免费一区二区三区| 中文字幕人妻熟女乱码| 色婷婷av一区二区三区视频| 免费久久久久久久精品成人欧美视频| 黄片播放在线免费| 国产人伦9x9x在线观看 | 激情视频va一区二区三区| 女人被躁到高潮嗷嗷叫费观| 777久久人妻少妇嫩草av网站| 在现免费观看毛片| 色婷婷久久久亚洲欧美| 麻豆乱淫一区二区| 久久久久精品性色| 下体分泌物呈黄色| 国产片特级美女逼逼视频| 国产一区有黄有色的免费视频| 国产精品.久久久| 日韩中字成人| 日韩欧美精品免费久久| 欧美97在线视频| 午夜福利在线观看免费完整高清在| 女人精品久久久久毛片| 精品久久久精品久久久| 一区二区日韩欧美中文字幕| 只有这里有精品99| 高清不卡的av网站| 国产亚洲精品第一综合不卡| 人成视频在线观看免费观看| 精品一区在线观看国产| 亚洲欧美日韩另类电影网站| 中文字幕人妻丝袜一区二区 | 男女国产视频网站| 嫩草影院入口| 美女国产高潮福利片在线看| 国产成人a∨麻豆精品| 欧美日韩一级在线毛片| 日本av免费视频播放| 美女大奶头黄色视频| 男人操女人黄网站| 国产精品人妻久久久影院| 日韩成人av中文字幕在线观看| 日本欧美国产在线视频| 婷婷色av中文字幕| 精品亚洲成a人片在线观看| 久久久欧美国产精品| 亚洲av电影在线进入| 男女边摸边吃奶| av片东京热男人的天堂| 国产精品 欧美亚洲| 啦啦啦中文免费视频观看日本| 午夜福利视频精品| 夜夜骑夜夜射夜夜干| 建设人人有责人人尽责人人享有的| 乱人伦中国视频| 午夜免费男女啪啪视频观看| 国产高清不卡午夜福利| 国产黄色免费在线视频| 日本vs欧美在线观看视频| 青春草亚洲视频在线观看| 性色avwww在线观看| 韩国高清视频一区二区三区| 日本91视频免费播放| 久久久久国产网址| 最新的欧美精品一区二区| 久久久久视频综合| 国产片特级美女逼逼视频| 成人午夜精彩视频在线观看| 日本午夜av视频| 亚洲成人手机| 青春草视频在线免费观看| 伊人亚洲综合成人网| 伦理电影免费视频| 韩国精品一区二区三区| 成人毛片60女人毛片免费| 一级毛片电影观看| 亚洲国产欧美日韩在线播放| 国产免费现黄频在线看| 欧美中文综合在线视频| 国产欧美日韩一区二区三区在线| av.在线天堂| 亚洲内射少妇av| 一级a爱视频在线免费观看| 亚洲视频免费观看视频| 国产 一区精品| 啦啦啦在线免费观看视频4| av线在线观看网站| 亚洲av欧美aⅴ国产| 91久久精品国产一区二区三区| 国产亚洲一区二区精品| 99国产综合亚洲精品| 成年女人毛片免费观看观看9 | 国产精品人妻久久久影院| 国产黄色视频一区二区在线观看| 精品酒店卫生间| av片东京热男人的天堂| 国产精品一二三区在线看| 久久鲁丝午夜福利片| 色婷婷av一区二区三区视频| 97精品久久久久久久久久精品| 欧美黄色片欧美黄色片| 亚洲av男天堂| 丰满饥渴人妻一区二区三| 久久久久国产网址| 国产激情久久老熟女| 丰满饥渴人妻一区二区三| 五月开心婷婷网| 久久久久久伊人网av| 在现免费观看毛片| 久久久久国产网址| 国产男女内射视频| 久久久久久人人人人人| 亚洲av日韩在线播放| 国产片内射在线| 蜜桃国产av成人99| 肉色欧美久久久久久久蜜桃| 在线观看美女被高潮喷水网站| 99久久人妻综合| 香蕉丝袜av| 色哟哟·www| 亚洲av综合色区一区| 日韩熟女老妇一区二区性免费视频| av片东京热男人的天堂| 久久久精品区二区三区| av.在线天堂| 久久ye,这里只有精品| 如何舔出高潮| 人成视频在线观看免费观看| 毛片一级片免费看久久久久| 午夜免费男女啪啪视频观看| 色吧在线观看| 成人二区视频| 免费女性裸体啪啪无遮挡网站| 可以免费在线观看a视频的电影网站 | 侵犯人妻中文字幕一二三四区| av视频免费观看在线观看| 国产 一区精品| 国语对白做爰xxxⅹ性视频网站| 国产精品香港三级国产av潘金莲 | 美女大奶头黄色视频| 校园人妻丝袜中文字幕| 精品国产一区二区三区四区第35| 国产黄频视频在线观看| 国精品久久久久久国模美| 国产野战对白在线观看| 久久精品亚洲av国产电影网| 99久久精品国产国产毛片| 国产精品国产av在线观看| 国产国语露脸激情在线看| 我要看黄色一级片免费的| 午夜免费男女啪啪视频观看| 菩萨蛮人人尽说江南好唐韦庄| 五月天丁香电影| 亚洲在久久综合| 国产97色在线日韩免费| 欧美精品亚洲一区二区| 国产高清不卡午夜福利| 日本爱情动作片www.在线观看| videos熟女内射| 男女边摸边吃奶| 热re99久久国产66热| 秋霞伦理黄片| 色婷婷av一区二区三区视频| 亚洲欧美日韩另类电影网站| 亚洲欧美精品自产自拍| 国产免费视频播放在线视频| 久久亚洲国产成人精品v| 搡女人真爽免费视频火全软件| 男人爽女人下面视频在线观看| 捣出白浆h1v1| 久久久精品94久久精品| 丁香六月天网| 哪个播放器可以免费观看大片| 国产精品久久久久成人av| 亚洲在久久综合| 国产男人的电影天堂91| 亚洲少妇的诱惑av| 亚洲色图 男人天堂 中文字幕| 99久久精品国产国产毛片| 亚洲成av片中文字幕在线观看 | 乱人伦中国视频| 亚洲av中文av极速乱| 国产成人a∨麻豆精品| 国产av一区二区精品久久| 久久久久精品久久久久真实原创| 日本欧美国产在线视频| 久久国产精品大桥未久av| 精品一区二区免费观看| 少妇猛男粗大的猛烈进出视频| 免费看av在线观看网站| 女的被弄到高潮叫床怎么办| 精品亚洲成a人片在线观看| 久久精品国产亚洲av高清一级| 久久久久久久久免费视频了| 人妻 亚洲 视频| 亚洲欧洲国产日韩| 亚洲成人一二三区av| 满18在线观看网站| 国产成人a∨麻豆精品| 蜜桃在线观看..| 成人国产av品久久久| 国产亚洲精品第一综合不卡| 中文字幕人妻熟女乱码| 精品国产超薄肉色丝袜足j| 中文字幕另类日韩欧美亚洲嫩草| 精品少妇内射三级| 99re6热这里在线精品视频| 日韩三级伦理在线观看| 女的被弄到高潮叫床怎么办| 18在线观看网站| 天天躁日日躁夜夜躁夜夜| 熟女av电影| 七月丁香在线播放| 久久精品国产亚洲av天美| 国产精品国产三级专区第一集| 亚洲欧美一区二区三区久久| 性高湖久久久久久久久免费观看| 最近中文字幕高清免费大全6| 久久这里有精品视频免费| 亚洲综合色惰| 丝瓜视频免费看黄片| 这个男人来自地球电影免费观看 | 亚洲av综合色区一区| 一本一本久久a久久精品综合妖精 国产伦在线观看视频一区 | 亚洲成人av在线免费| 最近最新中文字幕大全免费视频 | 秋霞在线观看毛片| 久久婷婷青草| 亚洲av国产av综合av卡| 久久ye,这里只有精品| 丝袜在线中文字幕| 亚洲成色77777| 色吧在线观看| 曰老女人黄片| 国产午夜精品一二区理论片| 丝袜人妻中文字幕| 日韩成人av中文字幕在线观看| 久久久久久久国产电影| 国产av精品麻豆| 久久久国产一区二区| 久久久久久人妻| 91国产中文字幕| 欧美日韩av久久| 国产有黄有色有爽视频| 99久久综合免费| 尾随美女入室| 国产片内射在线| 久久国产精品男人的天堂亚洲| 少妇被粗大猛烈的视频| 超碰97精品在线观看| 边亲边吃奶的免费视频| 天天躁日日躁夜夜躁夜夜| 久久鲁丝午夜福利片| 国产精品一区二区在线不卡| 久久精品aⅴ一区二区三区四区 | 亚洲av在线观看美女高潮| 香蕉国产在线看| a 毛片基地| 免费观看av网站的网址| 久久精品国产综合久久久| 欧美日本中文国产一区发布| xxxhd国产人妻xxx| 不卡视频在线观看欧美| 久久综合国产亚洲精品| 1024香蕉在线观看| 搡老乐熟女国产| 精品视频人人做人人爽| 国产一区亚洲一区在线观看| 国产野战对白在线观看| 色吧在线观看| 波多野结衣av一区二区av| 最新中文字幕久久久久| 久久人人97超碰香蕉20202| 亚洲美女黄色视频免费看| 777久久人妻少妇嫩草av网站| 黄频高清免费视频| 一级毛片我不卡| 看免费成人av毛片| 亚洲精品久久午夜乱码| 欧美人与善性xxx| √禁漫天堂资源中文www| 人成视频在线观看免费观看| 免费在线观看视频国产中文字幕亚洲 | 欧美日韩亚洲国产一区二区在线观看 | 欧美精品国产亚洲| 欧美97在线视频| 亚洲一级一片aⅴ在线观看| 精品99又大又爽又粗少妇毛片| 欧美日韩视频精品一区| 亚洲欧美精品综合一区二区三区 | 精品卡一卡二卡四卡免费| 日本黄色日本黄色录像| 亚洲 欧美一区二区三区| 亚洲男人天堂网一区| 成人影院久久| 午夜福利,免费看| 久久久久人妻精品一区果冻| 久久久欧美国产精品| 亚洲欧洲日产国产| 国产色婷婷99| 又粗又硬又长又爽又黄的视频| 天天躁狠狠躁夜夜躁狠狠躁| 中国三级夫妇交换| 亚洲欧美色中文字幕在线| av免费在线看不卡| 国产亚洲av片在线观看秒播厂| 国产麻豆69| 在线亚洲精品国产二区图片欧美| 美女主播在线视频| 女的被弄到高潮叫床怎么办| 成人国产麻豆网| 国产精品久久久久久久久免| 免费在线观看黄色视频的| 香蕉精品网在线| 青草久久国产| 在线观看www视频免费| 亚洲av中文av极速乱| 狂野欧美激情性bbbbbb| 视频在线观看一区二区三区| 建设人人有责人人尽责人人享有的| 国产黄色免费在线视频| 久久久精品区二区三区| 哪个播放器可以免费观看大片| 国产精品久久久久久精品古装| 汤姆久久久久久久影院中文字幕| 亚洲精品在线美女| 观看av在线不卡| 妹子高潮喷水视频| 波多野结衣一区麻豆| 激情视频va一区二区三区| 国产精品偷伦视频观看了| 国产精品一二三区在线看| 只有这里有精品99| 黄色 视频免费看| 69精品国产乱码久久久| 纯流量卡能插随身wifi吗| 一区在线观看完整版| 999精品在线视频| 一区在线观看完整版| 黄色 视频免费看| 99香蕉大伊视频| 999久久久国产精品视频| 日韩精品有码人妻一区| 国产野战对白在线观看| 国产成人精品婷婷| 久久毛片免费看一区二区三区| 欧美精品av麻豆av| 亚洲在久久综合| 最近中文字幕高清免费大全6| 在线观看免费视频网站a站| 国产成人91sexporn| 免费黄色在线免费观看| 深夜精品福利| 香蕉国产在线看| av不卡在线播放| 在线天堂中文资源库| 一级片'在线观看视频| 熟妇人妻不卡中文字幕| 国产黄色免费在线视频| 国产在视频线精品| 看十八女毛片水多多多| 亚洲国产精品一区三区| 日本欧美视频一区| 久久精品久久久久久噜噜老黄| 少妇熟女欧美另类| 大话2 男鬼变身卡| 亚洲成人手机| 99国产精品免费福利视频| 亚洲精品乱久久久久久| 美女视频免费永久观看网站| 国产成人精品久久二区二区91 | 国产精品三级大全| 精品亚洲成a人片在线观看| 69精品国产乱码久久久| 一区二区三区激情视频| 五月伊人婷婷丁香| 精品人妻一区二区三区麻豆| 成年女人毛片免费观看观看9 | 日本-黄色视频高清免费观看| 97在线视频观看| 美女国产高潮福利片在线看| 国产精品久久久av美女十八| 国产高清不卡午夜福利| 免费大片黄手机在线观看| 老司机影院毛片| 99热网站在线观看| 亚洲一区中文字幕在线| 亚洲男人天堂网一区| 中文字幕色久视频| 如何舔出高潮| 这个男人来自地球电影免费观看 | 日日爽夜夜爽网站| 成人午夜精彩视频在线观看| 国产成人91sexporn| 黄色 视频免费看| 国产女主播在线喷水免费视频网站| 日韩免费高清中文字幕av| 国产欧美亚洲国产| √禁漫天堂资源中文www| 一区二区三区激情视频| 亚洲国产日韩一区二区| 激情五月婷婷亚洲| 国产精品久久久av美女十八| 久久久久久伊人网av| 狠狠精品人妻久久久久久综合| 久久精品亚洲av国产电影网| 五月天丁香电影| 老司机影院毛片| 国精品久久久久久国模美| 日韩欧美一区视频在线观看| 最近最新中文字幕免费大全7| 老汉色∧v一级毛片| 少妇的丰满在线观看| a级毛片黄视频| 肉色欧美久久久久久久蜜桃| 男男h啪啪无遮挡| 1024香蕉在线观看| 有码 亚洲区| 国产xxxxx性猛交| 爱豆传媒免费全集在线观看| 在线免费观看不下载黄p国产| 亚洲经典国产精华液单| 国产精品免费大片| 亚洲精品中文字幕在线视频| 电影成人av| 在线 av 中文字幕| 亚洲欧美一区二区三区黑人 | 免费看不卡的av| 菩萨蛮人人尽说江南好唐韦庄| 在线亚洲精品国产二区图片欧美| 午夜日本视频在线| 中文乱码字字幕精品一区二区三区| 叶爱在线成人免费视频播放| 人人妻人人添人人爽欧美一区卜| 少妇的逼水好多| 午夜激情久久久久久久| 亚洲av在线观看美女高潮| 国产一区二区三区av在线| 婷婷色综合www| 久久久久国产网址| av免费在线看不卡| 精品亚洲成a人片在线观看| 一区在线观看完整版| 国产黄色视频一区二区在线观看| 久久精品国产亚洲av高清一级| 久久午夜福利片| 日本黄色日本黄色录像| 黑人猛操日本美女一级片| 男女午夜视频在线观看| 永久免费av网站大全| 大片免费播放器 马上看| 亚洲精品一二三| √禁漫天堂资源中文www| 久久精品国产鲁丝片午夜精品| 人人妻人人澡人人看| 国产精品久久久久久精品古装| 只有这里有精品99| 中文乱码字字幕精品一区二区三区| 熟女电影av网| 午夜日韩欧美国产| 看免费成人av毛片| 最近中文字幕高清免费大全6| 尾随美女入室| 青春草亚洲视频在线观看| 亚洲精品一二三| 久久影院123| 精品人妻在线不人妻| av又黄又爽大尺度在线免费看| 亚洲欧美色中文字幕在线| 免费不卡的大黄色大毛片视频在线观看| av在线app专区| 色婷婷av一区二区三区视频| 极品少妇高潮喷水抽搐| 日韩一卡2卡3卡4卡2021年| 亚洲精品一区蜜桃| 国产一区二区在线观看av| 亚洲av在线观看美女高潮| 久久久精品国产亚洲av高清涩受| 欧美成人午夜免费资源| 视频区图区小说| 免费黄网站久久成人精品| 成人国产av品久久久| 日本wwww免费看| 国产亚洲一区二区精品| 亚洲精品自拍成人| 2021少妇久久久久久久久久久| 久久久国产精品麻豆| 天美传媒精品一区二区| 1024香蕉在线观看| 91久久精品国产一区二区三区| 久热这里只有精品99| 爱豆传媒免费全集在线观看| videos熟女内射| 最新中文字幕久久久久| 国产精品国产三级国产专区5o| 亚洲第一区二区三区不卡| 国产成人a∨麻豆精品| 国产精品.久久久| 国精品久久久久久国模美| 欧美+日韩+精品| 欧美日韩视频高清一区二区三区二| 国产精品不卡视频一区二区| 国产片内射在线| 成人午夜精彩视频在线观看| 一级片'在线观看视频| 国产色婷婷99| 自拍欧美九色日韩亚洲蝌蚪91| 久久国产精品男人的天堂亚洲| av网站在线播放免费| 亚洲精品久久成人aⅴ小说| 丰满乱子伦码专区| 免费看不卡的av| 黄片播放在线免费| 亚洲国产日韩一区二区| 美女福利国产在线| 在线天堂最新版资源| 在线 av 中文字幕| 亚洲精品aⅴ在线观看| 2021少妇久久久久久久久久久| 国产女主播在线喷水免费视频网站| av女优亚洲男人天堂| 青草久久国产| 纯流量卡能插随身wifi吗| 女人精品久久久久毛片| 美女中出高潮动态图| 久久久久人妻精品一区果冻| 十八禁网站网址无遮挡| 少妇猛男粗大的猛烈进出视频| 国产熟女欧美一区二区| av电影中文网址| 亚洲av日韩在线播放| 精品少妇一区二区三区视频日本电影 | 国产精品嫩草影院av在线观看| 欧美精品人与动牲交sv欧美| 国产亚洲一区二区精品| 国产极品粉嫩免费观看在线| 老熟女久久久| 可以免费在线观看a视频的电影网站 | 久久久久网色| 1024视频免费在线观看| 交换朋友夫妻互换小说| 女性被躁到高潮视频| 日本-黄色视频高清免费观看| 色婷婷久久久亚洲欧美| 超色免费av| 欧美成人午夜免费资源| 日本vs欧美在线观看视频| 一个人免费看片子| 日本猛色少妇xxxxx猛交久久| 免费在线观看完整版高清| 久久 成人 亚洲| 免费观看av网站的网址| 人妻少妇偷人精品九色| 国产精品 国内视频| 韩国高清视频一区二区三区| 99香蕉大伊视频| 欧美av亚洲av综合av国产av | 日韩一卡2卡3卡4卡2021年| 波多野结衣av一区二区av| 国产精品三级大全| 在线精品无人区一区二区三| 一区二区三区四区激情视频| 亚洲人成77777在线视频| 老司机亚洲免费影院| 91午夜精品亚洲一区二区三区| 亚洲精品在线美女| 波多野结衣一区麻豆| 亚洲美女搞黄在线观看| 少妇人妻久久综合中文| av在线老鸭窝| 大片电影免费在线观看免费| 久热这里只有精品99| 黑人欧美特级aaaaaa片| 亚洲精品美女久久久久99蜜臀 | 夫妻性生交免费视频一级片| videossex国产| 亚洲精品aⅴ在线观看| 丰满饥渴人妻一区二区三|